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Abstract 

Federal  procurement  has  typically  focused  much  of  its  efforts  on  the  acquisition 
of  products,  with  acquisition  of  services  not  being  held  in  as  high  of  regard.  However, 
recent  changes  have  required  the  Department  of  Defense  and  the  US  Air  Force  (USAF)  to 
increase  spending  on  services.  This  significant  increase  in  spending  on  services  suggests 
a  need  to  ensure  acquisition  professionals  are  prepared  to  apply  sound  business  judgment 
to  Performance  Based  Service  Acquisition  (PBSA)  strategies.  PBSA  involves  acquisition 
strategies,  methods,  and  techniques  that  describe  and  communicate  measurable  outcomes 
rather  than  direct  performance  processes.  The  Secretary  of  Defense,  in  order  to  ensure 
the  success  of  PBSA,  has  established  a  goal  of  50%  of  all  service  acquisitions  meet 
PBSA  requirements  by  2005. 

This  thesis  looks  the  current  status  of  PBSA  with  the  USAF,  if  goals  are  being  met, 
are  there  any  trends  to  suggest  future  usage  of  PBSA,  and  what  factors,  if  any,  are  related 
to  whether  a  contract  is  classified  as  having  been  awarded  using  PBSA  strategies.  A 
combination  of  demographics,  nominal  logistical  regression,  and  contingency  tables  will 
used  in  order  to  determine  if  the  USAF  is  in  compliance  and  if  there  are  correlations 
within  PBSA  that  affect  its  use.  The  results  of  this  thesis  will  serve  as  a  starting  point 
from  which  further  research  can  develop  and  provide  information  that  can  help  utilize 
PBSA  in  the  future. 
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PERFORMANCE  BASED  SERVICE  ACQUISTION  (PBSA): 


A  DYNAMIC  LOOK  AT  PBSA  IN  THE  AIR  FORCE 


I.  Introduction 


Overview 

The  Department  of  Defense  is  increasingly  relying  on  the  acquisition  of  services 
to  meet  its  mission  needs.  Over  the  last  couple  of  years,  we  have  made  some 
important  strides  in  improving  the  quality  of  our  acquisition  of  services.  We  have 
new  policy  regarding  performance  based  service  acquisitions  requiring  that  50% 
of  all  service  acquisition  must  meet  the  Performance  Based  Services  Acquisition 
standards  by  2005.  ...  Yet  we  have  not  achieved  the  level  of  excellence  and 
consistency  that  is  essential.  (Oliver,  2001) 


Background 

Federal  procurement  has  typically  focused  much  of  its  efforts  on  the  acquisition 
of  products,  with  acquisition  of  services  not  being  held  in  as  high  of  regard.  In  January 
of  2001,  Dr.  Gansler,  then  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics,  signed  the  Guidebook  for  Performance  Based  Service  Acquisition  (PBSA)  in 
the  Department  of  Defense  (DoD).  This  guidebook  was  necessitated  by  the  large 
increase  in  Department  of  Defense  procurement  of  services,  $39. 9B  to  $51.8B,  from 
1992-1999.  In  1999,  services  equaled  the  total  dollars  spent  on  supplies  and  systems 
(Gansler,  2001). 

PBSA  involves  acquisition  strategies,  methods,  and  techniques  that  describe  and 
communicate  measurable  outcomes  rather  than  direct  perfonnance  processes.  It  is 
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structured  around  defining  a  service  requirement  in  tenns  of  perfonnance  objectives  and 
providing  contractors  the  latitude  to  determine  how  to  meet  those  objectives.  Simply  put, 
PBS  A  is  a  method  for  specifying  what  is  required  and  then  placing  the  responsibility  or 
burden  for  how  it  is  accomplished  on  the  contractor.  (Gansler,  2000) 

The  Federal  Government  has  emphasized  the  importance  of  PBS  A  in  a  stream  of 
direction  dating  from  early  1991,  when  the  Office  of  Federal  Procurement  Policy  (OFPP) 
signed  Policy  Letter  91-2  to  institute  a  service  contracting  policy  that  emphasized  the  use 
of  performance  requirements  and  quality  standards  (OFPP,  1991).  Subsequent  emphasis 
included  direction  from  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  that  mandated  the  use  of  PBSA  “wherever  possible”  (Gansler,  2000);  and 
inclusion  in  the  President’s  vision  for  better  Government  and  as  an  objective  in  the 
Procurement  Executives  Council’s  2001-2005  Strategic  Plan.  DoD’s  goal  is  to  increase 
the  use  of  PBSA  to  acquire  best  value  services  with  the  objective  of  attaining  a  minimum 
of  50  percent  of  eligible  service  dollars  awarded  as  PBSAs  by  Fiscal  Year  2005  (Gansler, 
2000). 

With  increasing  dollar  amounts  being  spent  on  service  related  items,  it  is 
imperative  that  the  objectives  for  PBSA  be  understood.  The  Guidebook  for  Performance- 
Based  Service  Acquisition  (2001)  lists  five  objectives: 

1.  Maximize  Performance.  This  objective  allows  a  contractor  to  deliver  the 
required  service  by  following  its  own  best  practices.  The  focus  is  the  end 
result;  contractors  can  adjust  their  processes  through  the  life  of  the  contract, 
without  the  burden  of  contract  modifications,  provided  that  the  delivered 


2 


service  remains  in  accordance  with  the  contract.  Incentives  can  further 


motivate  contractors  to  furnish  greater  results. 

2.  Maximize  Competition  and  Innovation.  This  objective  encourages  innovation 
from  suppliers,  since  government-directed  solutions  are  no  longer  mandated. 
In  order  to  maximize  competitive  alternatives,  performance  requirements  are 
used  instead,  increasing  innovation  and  attracting  a  broader  industry  base. 

3.  Encourage  and  Promote  the  Use  of  Commercial  Services.  Use  of  procedures 
from  the  Federal  Acquisition  Regulation  (FAR)  Part  12  (Acquisition  of 
Commercial  Items)  minimizes  the  reporting  burden  and  reduces  the  use  of 
government-unique  contract  clauses  and  similar  requirements.  The  use  of 
commercial  services  offers  the  additional  anticipated  benefit  of  attracting  a 
broader  industry  base. 

4.  Shift  in  Risk.  This  objective  shifts  acquisition  risks  from  the  Government  to 
industry.  Contractors  are  responsible  for  achieving  objectives  in  the  statement 
of  work  through  use  of  their  own  best  practices.  Agencies  should  consider 
this  shift  in  risk  in  determining  the  appropriate  acquisition  incentives. 

5.  Achieve  Savings.  Experiences  in  both  industry  and  government  have 
demonstrated  that  use  of  perfonnance  requirements  result  in  cost  savings. 

Research  Objective 

The  purpose  of  this  thesis  is  to  understand  possible  influences  for  the  decision  to 
use  PBSA  strategies  on  a  specific  acquisition.  The  study  will  investigate  possible 
correlations  between  whether  a  contract  is  classified  as  having  been  awarded  using  PBSA 
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strategies  and  such  key  factors  as  type  of  services,  dollar  amount,  length  of  contract, 
business  size,  and  contract  type.  Since  Dr.  Gansler’s  (2001)  direction  to  use  PBSA 
“wherever  possible”  is,  at  the  time  of  this  study  only  a  few  years  old,  there  has  been  little 
time  for  this  issue  to  be  addressed  within  the  acquisition  community.  This  exploratory 
study  will  serve  as  a  starting  point  from  which  further  research  can  develop.  This  study 
will  focus  on  contracts  awarded  within  the  USAF,  and  specifically  within  Air  Force 
Material  Command  (AFMC).  The  data  from  AFMC  represents  approximately  60%  of 
the  US  Air  Force’s  budget,  represents  a  large  sample  of  USAF  obligations,  and  is  readily 
obtainable. 

Thesis  Structure 

The  remainder  of  the  thesis  is  organized  as  follows:  Chapter  2  will  provide  a  review 
of  the  relevant  literature,  including  a  discussion  of  PBSA  within  the  DoD.  Chapter  3  will 
focus  on  the  research  methodology  aspects  to  be  employed  in  conducting  this  research 
effort.  Chapter  4  will  provide  data  analysis  and  results.  Finally,  Chapter  5  will  provide 
conclusions  and  recommendations  for  further  research. 
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II.  Literature  Review 


This  chapter  describes  the  literature  relevant  to  Perfonnance  Based  Service 
Acquisition  (PBSA)  within  the  Federal  Government,  the  Department  of  Defense  (DoD), 
the  US  Air  Force  (USAF),  and  the  objectives  that  are  established  for  PBSAs.  The 
chapter  will  include  basic  definitions,  a  description  of  law  and  public  policies,  a  synopsis 
of  regulations,  DoD  and  USAF  guidance,  and  previous  research  and  findings  conducted 
by  the  RAND  corporation  regarding  PBSA.  Finally,  the  chapter  concludes  with  a 
discussion  of  current  transition  of  the  Federal  Government  and  the  DoD  to  PBSA  and 
some  issues  related  to  this  transition. 

Definitions,  Law,  Public  Policies,  and  Federal  Regulations 

The  key  to  understanding  PBSA  lies  within  the  definitions,  laws,  public  policies, 
and  regulations  that  build  this  arena.  A  brief  history  of  some  of  the  key  acts,  policies,  and 
regulations  that  have  influenced  PBSA  over  the  years  illustrates  the  transformation  from 
traditional  services  contracting  to  PBSA  and  the  transfonnation’s  impact  on  the  entire 
procurement  process. 

PBSA  involves  an  acquisition  that  is  centered  telling  the  contractor  what  the 
Government  requires,  without  telling  them  how  to  do  it  (Gansler,  2000).  Federal 
Acquisition  Regulation  (FAR)  Part  37  provides  general  guidance  for  the  acquisition  and 
management  of  services  by  contract.  FAR  Part  37  requires  the  use  of  PBSA  to  the 
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maximum  extent  possible  and  lays  out  policies  and  procedure  to  support  this  goal.  FAR 

Part  37.101  defines  a  “service  contract”  as: 

...  a  contract  that  directly  engages  the  time  and  effort  of  a  contractor 
whose  primary  purpose  is  to  perform  an  identifiable  task  rather  than  to 
furnish  an  end  item  of  supply.  . . .  Some  of  the  areas  in  which  service 
contracts  are  found  include  the  following: 

(1)  Maintenance,  overhaul,  repair,  servicing,  rehabilitation,  salvage, 

modernization,  or  modification  of  supplies,  systems,  or  equipment. 

(2)  Routine  recurring  maintenance  of  real  property. 

(3)  Housekeeping  and  base  services. 

(4)  Advisory  and  assistance  services. 

(5)  Operation  of  Government-owned  equipment  facilities,  and  systems. 

(6)  Communications  services. 

(7)  Architect-Engineering. 

(8)  Transportation  and  related  services. 

(9)  Research  and  development. 

FAR  Part  37. 102  excludes  the  use  of  Architect  &  Engineering  services,  construction, 
utility  services,  and  services  that  are  incidental  to  supply  purchases  as  PBSA.  It  is 
important  to  note  that  FAR  Part  37  indicates  that  when  there  are  inconsistencies  or 
conflicts  between  Part  37  and  other  sections  of  the  FAR,  specifically  Parts  35  (R&D 
Contracting),  36  (Construction  and  Architect  &  Engineering),  39  (Information  and 
Technology),  and  47  (Transportation),  that  Part  37  is  superseded  by  the  conflicting  FAR 
part.  Despite  this  attempt  at  clarification,  problems  can  still  arise  from  unintentional 
vagueness  or  inconsistencies  among  FAR  parts.  (FAR  Part  37) 

FAR  Part  37.6  specifically  prescribes  the  policies  and  procedures  for  the  use  of 
PBSAs.  This  section  specifies  the  requirements  for  language  within  the  contract  and  how 
the  Statement  of  Work  (SOW)  is  to  be  written,  and  promotes  the  use  of  perfonnance 
incentives.  Further,  while  it  does  not  limit  PBSAs  to  specific  type  of  contract,  FAR  Part 
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37.6  does  promote  the  use  of  fixed-price  type  contracts  as  being  appropriate  for  services 
that  can  be  defined  objectively  (FAR  Part  37). 

Public  Law  93-400  established  the  Office  of  Federal  Procurement  Policy  (OFPP) 
in  1974  with  the  intent  of  having  one  office  promulgating  Government- wide  procurement 
policy  (Public  Law  93-400,  1974).  The  first  policy  to  affect  PBSA  was  issued  in  1991. 
Policy  Letter  91-2,  issued  to  the  Heads  of  Executive  Agencies  and  Departments  of 
Government,  emphasized  the  use  of  perfonnance  requirements  and  quality  standards  in 
defining  contract  requirements,  source  selections,  and  quality  assurance  for  the 
acquisition  of  services  (OFPP,  1991). 

OFPP  Policy  Letter  92-1  on  inherently  Governmental  Functions  establishes  policy 
relating  to  service  contracting  and  inherently  governmental  functions.  Its  purpose  is  to 
assist  Federal  Departments  and  agencies  in  avoiding  an  unacceptable  transfer  of  official 
responsibility  to  Government  contractors.  Contractors,  when  properly  used,  provide  a 
wide  variety  of  useful  services  that  play  an  important  part  in  helping  agencies  to 
accomplish  their  missions.  Agencies  use  service  contracts  to  acquire  special  knowledge 
and  skill  not  available  in  the  Government,  to  obtain  cost  effect  services,  or  to  obtain 
temporary  or  intermittent  services.  Contractors  may  not,  however,  perform  all  functions. 
It  is  clear  that  certain  functions  such  as  infantry  and  combat  troops  are  inherently 
Governmental  and  may  not  be  contracted.  On  the  other  hand,  it  is  also  clear  that  certain 
functions  such  as  building  maintenance,  food  services  operations,  and  secretarial  services 
are  not  inherently  Governmental  and  may  be  contracted.  The  difficulty  is  in  detennining 
which  services  are  or  are  not  inherently  Governmental.  (OFPP,  1992) 
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The  last  of  these  key  OFPP  Policy  letters,  Policy  Letter  93-1,  established 
Government -wide  policy,  assigned  responsibilities,  and  provided  guiding  principles  for 
Executive  Departments  and  agencies  in  managing  the  acquisition  and  use  of  services. 

This  policy  letter  provides  agencies  with  a  more  results-oriented  approach  to  managing 
and  administering  service  contracts  through  the  “best  practices”  concept.  (OFPP,  1994) 

The  OFPP  policy  letters  discussed  were  enacted  in  order  to  clarify  and  standardize 
service  contracting  practices.  All  three  policy  letters  build  towards  the  requirements  that 
are  laid  out  in  the  premise  of  PBS  A.  OFPP  Policy  Letter  91-2  is  the  basic  layout  to  a 
service  contract,  while  OFPP  Policy  Letter  91-2  begins  to  define  service  contracts  that  are 
and  are  not  acceptable  (OFPP,  1991;  OFPP,  1992).  Tying  in  OFPP  Policy  Letter  93-1 
and  the  use  of  best  practices,  along  with  the  previous  two  policy  letters,  we  can  begin  to 
see  the  basic  structure,  basis,  and  guidance  leading  to  PBSA  (OFPP,  1994). 

Public  Law  103-226,  the  Federal  Workforce  Restructuring  Act  of  1994,  came  into 
effect  because  of  the  downsizing  initiative  of  the  Federal  workforce  in  the  early  1990s. 
Public  perception  was  that  agencies  were  replacing  lost  employees  with  service  contract 
workers.  The  Federal  Workforce  Restructuring  Act  prohibits  the  use  of  service 
contracting  to  compensate  for  downsizing,  unless  contracted  services  provide  the 
government  a  demonstrable  cost  benefit  (National  Perfonnance  Review  Report,  1996). 

The  National  Defense  Authorization  Act  (NDAA)  for  Fiscal  Year  (FY)  2002, 
Public  Law  107-107,  Section  801-803  is  the  first  documentation  by  law  to  require  that  the 
DoD  meet  the  50%  requirement  of  PBSA  of  services  by  FY  2005  (Aldridge,  2002). 
Section  801  of  the  NDAA  for  FY  2002,  covers  management  of  the  procurement  of 
services  and  provides  basic  definitions.  More  importantly,  it  includes  the  requirement  for 
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data  collection,  including:  service  purchased,  total  dollars,  type  of  contract,  business  size, 
and  the  extent  of  competition  (NDAA,  2002).  This  lays  the  ground  work  for  future 
analysis  to  detect  trends  that  could  develop  within  PBSA. 

Section  802  of  the  NDAA  establishes  performance  goals  by  fiscal  year  for  PBSA 
both  on  a  competitive  basis  and  by  firm-fixed  price  (FFP)  type  contract.  Section  802 
requires  not  less  then  40%  of  contracts  be  competed  for  FY  2003,  50%  for  FY  2004,  and 
75%  by  FY  2011.  For  FFP  type  contracts,  Section  802  requires  not  less  then  25%  of 
contracts  are  FFP  for  FY  2003,  35%  for  FY  2004,  50%  for  FY  2005,  and  70%  by  FY 
2011  (NDAA,  2002).  Perfonnance  against  these  goals  must  be  analyzed  starting  with  FY 
2003  and  through  FY  2011.  Finally,  Section  803  discusses  the  use  of  competition  in  all 
multiple  award  contracts  above  $100,000  (NDAA,  2002).  Again  we  can  see  the 
emphasis  that  is  being  placed  on  competition. 

Though  the  OFPP  had  not  added  any  significant  policy  since  1994  regarding 
service  type  contracts,  the  OFPP  asserted  itself  in  March  2003  with  the  establishment  of 
an  Interagency  Task  Force  on  PBSA.  The  idea  behind  the  task  force  was  to  reinvigorate 
the  use  of  PBSA  and  capitalize  on  the  competitiveness  and  innovation  that  occurs  when 
contractors  are  given  the  freedom  to  utilize  best  business  practices  and  solutions  to  meet 
the  government’s  needs.  The  results  from  the  task  force  are  not  designed  to  show 
effectiveness  of  PBSA,  but  will  able  to  be  used  as  a  gauge  as  to  whether  PBSA  is  a 
priority  within  differing  agencies  (Styles,  2003). 

The  Interagency  Task  Force  on  PBSA  published  its  results  in  July,  2003.  The 
group  focused  on  making  recommendations  for  modifying  the  FAR  in  order  to  promote 
flexibility  for  the  use  of  PBS  As,  modifications  for  reporting  PBSA,  and  improving  the 
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availability  of  guidance.  While  the  task  force  made  several  recommendations  to  support 
the  key  areas  listed,  it  also  added  an  additional  discussion  on  cost  savings.  Specifically, 
the  task  force  suggests  that  it  is  possible  for  cost  savings  to  occur  by  utilizing  PBSA. 
However,  they  warn  that  there  is  little  data  to  support  any  assertion  of  savings  and  that  if 
the  data  were  available  and  sufficient,  it  would  be  difficult  to  isolate  the  reason  behind 
the  savings.  (Interagency  Task  Force,  2003) 

PBSA  definitions,  laws,  public  policies,  and  regulations  which  build  the 
framework  for  the  situation  around  what  the  Federal  Government  must  contend  with. 
However,  there  are  additional  requirements  that  build  upon  PBSA  requirements  when 
looking  at  the  DoD  and  USAF.  We  will  now  look  at  the  key  policies  and  guidance  that 
shape  the  DoD  and  USAF’s  view  on  PBSA. 


DoD  and  USAF  Guidance 

Air  Force  Instruction  (AFI)  63-124  was  released  into  the  USAF  acquisition 
community  on  1  April,  1999.  The  intent  of  AFI  63-124  was  to  replace  previous  policy 
for  service  contracts,  under  Air  Force  Manual  64-108,  in  an  effort  to  promote  PBSA 
throughout  the  USAF.  AFI  63-124  defined  four  criteria  (in  accordance  with  FAR  PART 
37.6)  that  must  be  met: 

(1)  Describe  requirements  in  terms  of  results  required  rather  than  the  methods 
of  performance  of  the  work. 

(2)  Use  the  measurable  performance  standards  and  quality  assurance 
surveillance  plans. 

(3)  Specify  procedures  for  reduction  of  fee  or  reduction  to  price  of  a  fixed-price 
contract  when  services  are  not  performed  or  do  not  meet  contract 
requirements. 

(4)  Include  performance  incentives  when  appropriate. 
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AFI  63-124  defined  key  rolls  and  responsibilities  within  PBSA;  defined  the  establishment 
for  support  and  working  teams;  and  sets  forth  requirements  for  the  SOW,  performance 
measures,  and  quality  assurance.  It  also  listed  services  that  are  exempt  from  falling  under 
PBSA,  in  addition  to  those  listed  under  FAR  Part  37.102.  (AFI  63-124,  1999) 

Effective  9  February  2004,  the  USAF  issued  an  Interim  Change  to  AFI  63-124. 
The  largest  change  was  the  withdrawal  of  services  exempt  from  PBSA  under  AFI  63-124. 
This  allowed  for  all  services  contracts  over  the  Simplified  Acquisition  Threshold  (SAT), 
currently  listed  at  $100,000  per  year,  to  be  eligible  for  PBSA  (IC2004-1,  2004).  Note, 
the  SAT  is  a  dollar  limitation  that  has  been  set  the  Federal  Government,  in  which 
simplified  acquisition  procedures  can  be  used  in  order  to  reduce  costs,  promote 
efficiency,  and  avoid  unnecessary  burden  (FAR  Part  13).  Only  those  services  still  listed 
under  FAR  Part  37.102  remain  exempt  from  PBSA  (IC2004-1,  2004).  This  change  will 
significantly  influence  achieving  the  goal  of  50%  PBSA  by  FY  2005. 

In  an  effort  meet  the  requirements  set  by  Dr.  Gansler  in  April  2000,  the  USAF 
issued  its  own  PBSA  Implementation  Plan  in  June  2000  (Gansler,  2000).  The  plan  laid 
out  guidance  for  types  of  services  that  would  be  acquired  under  PBSA  including: 
maintenance,  repair,  operations  and  support,  modifications,  modernization,  and  medical 
services.  Service  not  covered  by  the  plan  included  Research  and  Development  (R&D), 
most  Advisory  and  Assistance  Services  (A&AS),  and  items  excluded  within  Attachment 
2  of  AFI  63-124,  such  as  utilities,  Architect  &  Engineering,  and  services  incidental  to 
supply  purchases  (PBSA  USAF  Implementation  Plan,  2000;  AFI  63-124,  1999).  The 
justification  for  excluding  R&D  and  A&AS  is  due  to  the  type  of  contract  that  is  typically 
associated  with  this  type  of  service,  that  of  a  cost  reimbursement.  Cost  type  contracts, 
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while  possible  to  use  for  PBSA,  typically  do  not  define  outcomes  based  on  perfonnance 
or  definable  goals  due  to  the  nature  of  the  work  involved  and  typically  poor  candidates 
for  PBSA  (PBSA  USAF  Implementation  Plan,  2000).  The  final  exclusion  category, 
beyond  what  has  previously  been  stated,  affected  those  services  below  the  $100,000 
threshold,  matching  that  of  AFI  63-124,  but  encouraged  the  use  of  PBSA  where  possible 
below  this  threshold.  While  the  USAF  has  encouraged  the  use  of  PBSA  since  the 
issuance  of  AFI  63-124,  it’s  Implementation  Plan  set  fourth  the  requirement  of  tracking 
this  progress  beginning  in  Fiscal  Year  2001  (PBSA  USAF  Implementation  Plan,  2000). 

In  order  to  obtain  PBSA  strategies  and  what  Dr.  Gansler  labeled  as  “best 
commercial  practices,”  he  had  the  DoD  develop  the  Guidebook  for  PBSA,  which  was 
published  in  March  2001  (Gansler,  2001).  As  mentioned  previously,  the  guidebook 
states  objectives  that  the  Under  Secretary  of  Defense  would  like  to  obtain.  In  addition  to 
the  objectives,  the  guidebook  provides  guidance  consistent  with  FAR  and  OFPP  policy 
for:  market  research,  development  of  perfonnance  work  statements  and  measurable 
performance  standards,  incentives  and  remedies,  contractor  perfonnance  management, 
source  selection  considerations,  and  contract  administration  (Guidebook,  2001). 

The  latest  guidance  to  be  handed  down  through  the  DoD  and  USAF  is  the 
Management  and  Oversight  of  Acquisition  of  Services  Process  (MOASP).  This  guidance 
was  sent  out  in  April  2003  from  the  USAF  Program  Executive  Office  for  Services 
(AFPEO/SV),  now  the  AFPEO  for  Combat  &  Mission  Support  (CM),  and  primarily 
provides  written  guidance  to  reinforce  the  requirements  and  laws  mentioned  previously. 
Specifically,  the  MOASP  calls  attention  to  the  NDAA  for  FY  2002,  Pub.  Law  107-107, 
Section  801  and  DoD  Guidebook  for  PBSA  (MOASP,  2003) 
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Now  that  we  have  looked  at  guidance  through  the  DoD  and  USAF,  Federal 
policy,  and  law  for  PBSA,  we  have  a  solid  grasp  of  the  conditions  and  requirements 
leading  to  PBSA  usage  within  the  USAF.  Due  to  the  short  amount  of  time  the  PBSA  has 
been  required  by  the  USAF  and  Federal  Government,  little  has  been  accomplished  in 
looking  at  the  current  environment  and  how  it  is  progressing.  The  next  section  will  give 
a  look  at  what  research  has  been  accomplished  and  the  results  that  have  been  found. 

Previous  Research 

Project  AIR  FORCE.  Previous  research  on  PBSA  is  sketchy;  only  3  studies  were 
found.  The  USAF  contracted  the  RAND  Corporation  to  conduct  federally  funded  studies 
within  the  USAF.  A  division  of  RAND,  Project  AIR  FORCE,  provides  independent 
analyses  of  combat  readiness,  policy,  and  support  of  aerospace  forces  (Moore  et.al., 
2002).  From  Project  AIR  FORCE,  two  studies  have  specifically  focused  on  PBSA  while 
a  third  primarily  addressed  Supply  Chain  Management  but  included  portions  of  PBSA 
within  the  research.  The  results  were  released  in  the  form  of  documented  briefings. 

The  first  of  the  three  studies  is  titled,  “Performance-Based  Contracting  in  the  Air 
Force:  A  Report  on  Experiences  in  the  Field.”  This  first  look  into  PBSA  within  the  Air 
Force  studied  successful  examples  of  PBSA  implementation  and  to  see  how  to  apply 
them  within  other  areas.  The  research  was  conducted  on  22  contracts,  from  volunteer 
bases  that  identified  their  own  PBSA  contracts  as  successful.  Several  key  areas  were 
assessed  including:  training,  market  research,  defining  requirements,  SOW  and  quality 
assurance  plans,  contract  structure  (type),  source  selection  procedures,  and  past 
performance.  (Ausink  et.al.,  2001) 
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This  study  found  that  19  out  of  22  contracts  were  firm  fixed  type  contracts  and  21 
of  the  22  contracts  were  competitively  bid  with  4  of  them  awarded  under  FAR  Part  12, 
Commercial  Practices.  Additionally,  only  6  contracts  contained  incentive  fees.  The 
researchers  qualified  their  findings  by  stating  the  analysis  was  difficult  to  conduct,  and 
comparisons  were  hard  to  draw  between  old  and  new  contracts,  primarily  because  the 
scope  and  SOW  for  the  new  contract  were  different  from  the  previous  one.  That  being 
said,  the  research  found  it  difficult  to  draw  any  cost  comparisons  or  savings  due  to  the 
infancy  of  the  PBSA  contracts  and  change  in  scope  and  SOW.  Recommendations 
concluded  that  better  guidance  be  established  for  PBSA  within  the  USAF  and  that  better 
data  collection  would  be  required  in  order  to  detennine  if  changes  were  making  any 
impact  in  cost  or  performance.  (Ausink  et.al.,  2001) 

The  second  study  involving  PBSA  is  titled,  “Implementing  Best  Purchasing  and 
Supply  Management  Practices:  Lessons  from  Innovative  Commercial  Finns.”  While  the 
bulk  of  the  of  the  research  was  focused  on  Purchasing  and  Supply  Management  (PSM), 
the  researchers  identified  that  PBSA  includes  adaptations  of  best  commercial  PSM 
practices  and  that  the  USAF  considers  PBSA  as  having  a  more  narrow  focus  then  PSM. 
To  this  end,  interviews  were  conducted  and  included  gathering  information  on 
implementation  of  PBSA  within  the  USAF.  The  basic  findings  showed  that  while 
contracting  took  the  lead  role  in  making  PBSA  a  reality,  other  organizations,  such  as 
logistics  and  civil  engineering,  took  on  larger  roles  as  well,  in  order  for  PBSA  success. 
Secondary  to  this  finding,  is  that  in  order  to  understand  best  commercial  practices,  market 
research  and  benchmarking  will  play  key  factors.  Though  this  research  addressed 
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implementation  of  PBS  A,  no  hard  numbers  were  provided  as  to  a  status  of  the  increase  in 
PBSA  usage.  (Moore  et.al,  2002) 

The  final  RAND  study  involving  PBSA  is  titled,  “Implementing  Performance- 
Based  Service  Acquisition  (PBSA):  Perspectives  from  an  Air  Logistics  Center  and  a 
Product  Center.”  This  study  detennined  that  systems  type  contracts  had  different 
requirements,  which  allowed  for  infrequent  opportunities  for  performance  evaluations 
and/or  define  a  successful  outcome  for  evaluation.  The  researches  noted  it  was  difficult 
to  determine  whether  adding  performance  standards  improved  PBSA  use  because  the  use 
of  performance  standards  had  been  in  use  at  the  ALCs  prior  to  the  requirement  being 
formalized  in  AFI  63-124.  (Ausink  et.al.  2002) 

This  research  found  that  many  commercial  services  are  ideal  for  meeting  the 
requirements  of  AFI  36-124.  However,  services  supporting  systems  were  not  as  good  a 
fit  to  the  requirements  for  PBSA;  specifically,  meeting  the  second  requirement  of 
“measurable  performance  standards,”  was  extremely  difficult.  However,  the  researchers 
suggested  that  despite  the  difficulty,  systems  contracts  can  meet  the  intent  of  the  AFI, 
even  if  they  do  not  meet  the  second  criteria.  (Ausink  et.al.,  2002) 

While  there  has  been  limited  research  regarding  PBSA  within  the  DoD  and 
USAF,  research  in  the  civilian  community  seems  to  be  even  less.  Services  do  not  take  up 
as  much  of  commercial  industries’  budgets,  as  in  the  Federal  Government,  with  only  5- 
20%  range  for  the  contracting  of  external  services  (Barry,  2003).  There  are  some 
parallels  that  exist  between  USAF  and  the  commercial  sector  in  that  both  baseline 
requirements  (SOW),  provide  incentives,  and  develop  perfonnance  measures  (Barry, 
2003;  AFI  63-124).  While  it  is  possible  for  the  Federal  Government,  the  DoD,  or  the 
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USAF  to  obtain  the  goal  of  50%  of  all  services  to  be  PBSA  by  the  year  2005,  industry 
has  not  pushed  PBSA  to  this  level. 

Transition  to  PBSA 

As  shown  in  the  literature,  there  are  only  a  few  studies  that  have  been 
accomplished  regarding  PBSA  and  specifically  looking  at  the  USAF.  Key  elements 
regarding  the  type  of  contract  used,  business  size,  solicitation  procedures  and  competition 
requirements,  and  the  use  of  commercial  practices  have  all  been  addressed,  but  not  on  an 
in-depth  basis.  (Ausink  et.al.,  2001;  Ausink  et.al.,  2002) 

Conclusion 

This  chapter  described  the  relevant  literature  within  the  Federal  Government, 
DoD,  the  USAF,  and  the  objectives  that  are  established  for  PBSAs.  Basic  definitions,  a 
description  of  law  and  public  policies,  a  synopsis  of  regulations,  DoD  and  USAF 
guidance,  and  previous  research  and  findings  conducted  by  the  RAND  corporation 
regarding  PBSA  where  addressed.  The  chapter  concluded  with  a  discussion  of  current 
transition  of  Federal  Government  and  DoD  to  PBSA  and  some  issues  that  are  revolving 
around  this  transition.  These  areas  were  brought  together  into  developing  the  research 
objective.  Chapter  3  discusses  the  methodology  used  in  testing  the  research  objective  and 
the  data  obtained. 
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III.  Methodology 


The  purpose  of  this  chapter  is  to  explain  the  research  methods  that  were  used  to 
explore  the  possible  influences  into  Performance  Based  Service  Acquisition  (PBSA) 
strategies.  This  effort  takes  data  provided  by  Air  Force  Materiel  Command  (AFMC)  and 
looks  at  the  current  status  of  PBSA  within  AFMC  as  a  representation  of  the  overall  US 
Air  Force  (USAF)  PBSA  effort.  The  data  was  obtained  from  the  Department  of 
Defense’s  (DoD’s)  procurement  data  collection  (DD350),  which  feeds  this  information 
into  the  Federal  Procurement  Database  System  (FPDS). 

It  is  the  intent  of  this  study  to  produce  demographics  in  order  to  represent  current 
situation  regarding  PBSA  within  AFMC  as  a  representative  population  of  USAF  efforts. 
Additional  exploration  through  logistical  regression  analysis  will  be  conducted  in  order  to 
determine  if  there  is  any  correlation  between  certain  factors  within  a  contract  and  whether 
an  contract  action  is  classified  as  PBSA  or  not.  Finally,  contingency  table  analysis  will 
determine  if  there  is  dependency  between  elements  of  a  contract  and  whether  it  is 
classified  as  PBSA  or  not.  The  primary  goal  of  this  research  is  to  provide  a 
representation  of  the  status  of  PBSA  implementation  within  the  USAF  and  whether 
established  goals  for  PBSA  implementation  will  be  met.  Analysis  will  identify  any 
additional  factors  correlated  with  PBSA  implementation,  in  order  to  help  predict  future 
PBSA  actions  and  areas  that  can  possibly  be  improved  upon. 

This  chapter  includes  the  research  problem,  research  design,  the  instruments  used 
to  gather  data  and  the  reliability  of  that  data,  data  analysis,  and  concludes  with  a 
summary. 
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Research  Problem 


PBS  A  is  a  service  acquisition  tool  that  has  been  formally  utilized  within  the 
USAF  since  the  inception  of  Air  Force  Instruction  (AFI)  63-124,  on  1  April,  1999  (AFI 
63-124,  1999).  Dr.  Gansler  formally  set  a  goal  of  50%  of  all  service  contracts  within  the 
DoD,  which  includes  the  USAF,  to  be  PBSAs  by  Fiscal  Year  (FY)  2005  (Gansler,  2000). 
The  National  Defense  Authorization  Act  (NDAA)  for  FY  2002  set  this  goal  and  interim 
goals  for  PBSAs,  into  public  law  (NDAA,  2002).  Since  this  time,  very  little  research  has 
been  performed  regarding  PBSA  and  the  research  that  has  been  conducted,  has  had 
limited  results  in  its  findings. 

Because  of  the  limited  research  to  date,  this  exploratory  study  will  address  the 
following  research  question:  What  is  the  status  of  PBSA  implementation  in  the  US  Air 
Force  (USAF)?  Several  investigative  questions  were  developed  in  order  to  answer  the 
research  question: 

1 .  Are  the  goals,  as  established  by  Dr.  Gansler  and  NDAA,  being  met  by  AFMC  and 
the  USAF,  and  are  there  any  trends  to  suggest  future  usage  of  PBSA? 

2.  What  factors,  if  any,  are  related  to  whether  a  contract  is  classified  as  having  been 
awarded  using  PBSA  strategies? 

In  order  to  answer  the  research/investigative  questions  listed  above,  there  will  be  three 
primary  elements  involved;  demographics,  nominal  logistical  regression,  and  contingency 
tables.  To  address  this  research  problem,  the  following  research  design  was  developed. 
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Research  Design 

The  research  design  selected  for  this  study  was  a  combination  involving 
exploratory  and  causal-comparative  design.  This  design  was  selected  to  address  the 
focused  nature  of  this  topic,  the  effects  that  particular  areas  within  a  contract  interact  with 
one  another,  and  the  relatively  small  amount  of  research  that  has  been  conducted  in  this 
area.  Virtually  all  previous  studies  relating  to  PBSA  were  conducted  using  inductive 
research  and  qualitative  methods.  Inductive  reasoning  involves  making  generalized 
conclusions  about  a  population  being  studied,  while  qualitative  is  defined  as  data  that  can 
not  be  divided  or  measured  discretely,  usually  in  the  form  of  verbal  comments  or  field 
notes  (Leedy  and  Onnrod,  2001;  McClave  et.al.,  2001).  This  leads  to  the  ability  to  draw 
generalized  conclusions,  but  is  difficult  to  validate  the  results  in  concrete  tenns. 

This  study  will  explore  PBSA  from  a  different  perspective,  one  that  is  more 
quantitative  in  nature.  Quantitative  data  is  defined  as  data  that  can  be  divided  into 
discrete,  measurable  variables  that  are  numerically  represented,  such  as  money,  distance, 
or  weight  (Leedy  and  Onnrod,  2001).  The  literature  review  has  pointed  to  key  variables 
within  PBSA,  such  as  type  of  service,  dollar  amount  business  size,  and  the  contract  type. 
It  is  the  hope  that  these  key  elements  will  lead  to  correlations  within  PBSA,  but  there  has 
been  no  research  to  date  to  analyze  this.  This  has  led  to  the  need  for  research  that,  while 
exploratory,  either  confirms  or  disconfirms  these  correlations  without  preconceived 
conceptions  to  the  possible  findings  and  that  can  be  used  as  a  starting  platfonn  for  future 
research. 

The  causal-comparative  design  provides  a  means  by  which  we  can  examine  how 
specific  independent  variables  affect  the  dependent  variable  of  interests.  It  allows  a 
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backward  look  in  time,  with  the  intent  of  trying  to  determine  possible  relationships  or 
correlations,  and  attempts  to  explain  why  it  occurred.  Causal-comparative  design  is  not 
able  to  determine  cause  and  effect  since  a  researcher  does  not  have  control  over  the  data 
as  it  is  produced  (Leedy  and  Onnrod,  2001).  In  this  situation,  causal-comparative  design 
will  allow  us  to  see  if  there  are  any  correlations,  dependencies,  or  relationships  between 
whether  a  contract  action  is  classified  as  PBS  A  or  not,  compared  to  other  aspects  within  a 
contract  action. 

Data  Gathering  and  Reliability 

For  purposes  of  this  research,  data  gathered  through  the  DD350  system  will  be 
utilized.  The  DD350  system  gathers  individual  contracting  action  reports,  for  all  contract 
actions  classified  over  $25,000,  and  the  information  is  sent  for  Congressional  review 
(DFAR  253.204,  2003).  The  information  gathered  through  the  DD350  system  is  the  first 
level  of  input  from  a  contract  action  and  is  considered  primary  data.  Primary  data  is 
defined  as  data  that  is  closest  to  the  primary  source  of  information  or  closest  to  the  truth 
(Leedy  and  Onnrod,  2001).  Data  of  this  nature  is  considered  to  be  some  of  the  most 
reliable  and  valid  data  that  can  be  collected  and  utilized  for  research  (Leedy  and  Onnrod, 
2001). 

However,  no  data  collection  system  is  without  faults.  In  two  different  reports 
from  the  General  Accounting  Office  (GAO)  data  collected  by  the  DD350  was  found  to 
have  several  limitations.  These  limitations  include:  reporting  levels  that  exclude  millions 
of  dollars  in  procurement  obligations;  obligation  of  funds  versus  actual  expenditures;  lack 
of  data  for  subcontracts;  and  recorded  data  differing  from  actual  data  (GAO,  1998;  GAO; 
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2003).  Despite  these  limitations,  the  DD350  database  represents  the  greatest  collection  of 
possible  data  available  and  despite  the  systems  imperfections,  it  represents  a  more 
reliable  system  of  gathering  data  than  this  researcher  would  be  able  to  develop. 

Given  the  short  time  frame  that  the  USAF  has  used  PBS  A  there  is  limited  data  on 
this  subject.  The  data  available  in  the  DD350  system  includes  all  service  contract  actions 
for  FY  2002  for  the  AFMC  and  all  service  contract  actions  for  FY  2003  for  the  USAF. 

The  data  for  FY  2002  and  FY  2003,  will  be  utilized  in  the  demographic  portion  of  this 
research,  while  the  data  for  FY  2002  and  the  AFMC  portion  of  data  for  FY  2003, 
specifically  that  pertaining  to  AFMC,  will  be  utilized  for  logistical  regression  and 
contingency  tables.  Rationale  for  this  approach  is  provided  in  the  next  section. 

Data  Analysis 

Three  different  means  in  which  to  analyze  that  data  have  been  chosen  for  this 
research.  This  section  will  describe  each  methodology  and  why  it  was  chosen. 

Demographics.  Simple  demographics  will  be  used  to  answer  investigative 
question  1,  that  is,  to  determine  if  the  USAF  is  meeting  the  goal  established  for  FY  2003, 
twenty-five  percent  of  all  service  contract  actions  and  dollars  awarded  as  PBS  A  (NDAA, 
2002).  Demographics  also  provides  a  starting  point  for  analyzing  the  data  and 
determining  if  there  are  any  initial  trends  that  will  lead  to  further  testing  in  this  research. 

Utilizing  the  DD350  database  for  AFMC  and  the  USAF,  for  FY  2002  and  FY 
2003,  demographics  will  be  used  to  see  if  there  are  any  trends  that  are  developing  with 
regards  to  PBSA.  This  analysis  will  include  overall  actions  and  dollars  for  PBSA,  type  of 
contracts  utilized,  size  of  business  award  to,  and  percentage  of  competed  PBSA  contracts. 
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Besides  any  initial  trends  and  predominate  factors  that  are  identified,  the  use  of 
demographics  will  ultimately  answer  if  the  goals  for  PBS  A  are  being  achieved.  While 
demographics  will  answer  basic  questions,  it  will  not  answer  complex  correlations  or 
predictive  capabilities. 

Nominal  Logistical  Regression.  Nominal  logistical  regression  will  be  used  to 
answer  investigative  question  2.  The  data  being  utilized  for  this  particular  section  is 
strictly  the  data  that  pertains  to  AFMC.  Since  the  data  available  does  not  include  all  data 
points  for  the  USAF  for  FY  2002,  it  not  possible  to  run  regression  of  FY  2002  and  full 
database  of  FY  2003,  and  make  a  logical  comparisons.  The  data  between  the  two 
databases  would  skew  the  results  and  findings  would  not  be  relevant.  However,  by  the 
utilizing  a  portion  of  the  data  from  the  FY  2003  database,  specifically  that  of  AFMC,  a 
regression  model  can  be  run  across  two  consecutive  years  of  data.  AFMC  represents  the 
predominant  amount  of  contract  actions  and  dollars  spent  across  the  USAF, 
approximately  60%.  The  represents  a  good  sample  size,  with  sufficient  data  points,  in 
order  to  draw  conclusions  (Leedy  and  Ormrod,  2001). 

The  data  in  this  analysis  is  both  quantitative  and  qualitative/nominal  in  nature. 
Quantitative  data  is  defined  as  data  that  can  be  divided  into  discrete,  measurable  variables 
that  are  numerically  represented,  such  as  money,  distance,  or  weight  (Leedy  and  Ormrod, 
2001).  Conversely,  qualitative/nominal  data  is  defined  as  data  that  can  not  be  divided  or 
measured  discretely,  usual  in  the  form  of  verbal  comments  or  field  notes,  but  can  be 
assigned  a  discrete  value  with  no  implicit  order  (Leedy  and  Ormrod,  2001;  McClave 
et.al.,  2001).  In  this  situation,  the  dependent  variable  is  a  yes  or  no  response  as  to 
whether  or  not  a  contract  action  is  PBSA.  Since  yes  and  no  are  verbal  responses,  they  are 
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qualitative  in  nature,  however  they  can  be  described  in  nominal  form,  such  as  0  and  1 
(Leedy  and  Ormrod,  2001;  McClave  et.al.,  2001).  Many  of  the  variables  within  the  data 
set  that  the  research  will  be  analyzing  are  qualitative/nominal  or  multi-nominal,  responses 
resulting  in  numerical  assignment  of  0,  1,  2,  3,. . to  the  nth  number  (McClave  et.al., 
2001). 

When  the  response  variable  or  dependent  variable  is  nominal,  we  must  fit  the 
probabilities  that  the  response  is  one  of  r  different  response  levels  given  by  the  data 
values.  The  basic  model  is  that  for  each  observation: 

Probability  (Y  =  jth  response  level)  =  some  function  of  the  X  and  parameters 
The  simplest  model  for  a  nominal  response  in  which  the  probability  (p)  that  y,  the 
dependent  variable,  is  response  level  j  is  estimated  by  dividing  the  total  sample  count  n 
into  the  total  of  each  response  level  n/?  and  is  written: 

Pj  =  n/n 

All  other  models  are  compared  to  this  base  model.  The  base  model  serves  the  same  role 
for  a  nominal  response  as  the  sample  mean  does  for  continuous  models.  (McClave  et.al., 
2001) 

The  R  (U)  measures  the  portion  of  the  uncertainty  accounted  for  by  the  model 
and  is  calculated  by  taking  the  explained  sample  variability  divided  by  the  total  sample 
variability,  represented  as  follows: 

R  (U)  =  Explained  sample  variability/Total  sample  variability 
In  other  words,  this  will  explain  the  total  sample  variability  that  is  explained  by  the 
relationship  between  the  dependent  and  independent  variable(s).  An  R"  (U)  of  1  means 
that  the  factors  completely  predict  the  categorical  response.  An  R"  (U)  of  0  means  that 
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there  is  no  predictive  capability.  In  categorical  data  analysis,  high  R  values  are 
somewhat  rare.  (McClave  et.al.,  2001) 

Contingency  Tables.  As  a  secondary  check  of  this  information  and  the  nominal 
logistical  regression  process,  contingency  tables  have  been  chosen.  A  contingency  table 
is  a  statistical  model  used  for  multi-nominal  data  and  provides  a  determination  of 
dependence  (McClave  et.al.,  2001).  Contingency  table  analysis  provides  an  observed 
count  of  the  occurrence,  an  expected  count,  probability  for  each  cell,  row,  and  column, 
and  the  probability  chi-square  (X~)  for  determination  of  dependence.  See  Table  1  for  an 
example  of  a  contingency  table.  This  will  allow  for  a  comparison  to  the  findings  of 
nominal  logistical  regression,  identification  of  additional  correlations,  or  in  the  event  that 
no  relationships  have  been  found  through  previous  methods,  another  means  of  analyzing 
the  data.  This  approach  is  attempting  to  help  answer  investigative  question  2  of  this 
research. 

Like  that  of  nominal  logistical  regression,  the  data  that  will  be  used  for  this 
section  of  research  includes  the  AFMC  database  for  FY  2002  and  FY  2003.  Utilizing  this 
database  with  contingency  tables  is  a  secondary  method  of  determining  if  there  are 
correlations  between  factors  (also  known  as  classifications  for  contingency  tables)  within 
PBS  A. 
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Table  1.  Sample  Contingency  Table 


Dependent 

Variable  Y 

Totals 

1 

2 

3 

4 

1 

nn 

ni2 

ni3 

ni4 

ri 

P11 

P12 

P 1 3 

Pl4 

Prl 

Independent 

2 

n2i 

n22 

n23 

n24 

D 

Variable  X 

P21 

P22 

P23 

P24 

Pr2 

3 

n3i 

n32 

n33 

n34 

D 

P31 

P32 

P33 

P34 

Pr3 

Totals 

Cl 

C2 

C3 

c4 

n 

Pci 

Pc2 

Pc3 

Pc4 

1 

In  this  table,  n  represents  the  count  for  each  column  and  row.  For  instance,  n  1 2  represents 
the  cell  count  for  column  2,  row  1 .  r  and  c  represent  row  and  column  respectively,  while 
p  represents  the  probability  for  each  cell,  row,  or  column.  This  information  not  only  will 
give  a  general  breakout  of  the  data,  but  also  can  identify  whether  there  are  anomalies, 
utilizing  expected  and  actual  counts,  across  each  individual  cell.  In  addition  to  this,  chi- 
square  (X")  will  be  detennined  for  the  entire  contingency  table.  X"  is  the  sampling 
distribution  taken  from  the  comparison  of  the  sample  variance  to  the  hypothesized  value 
of  the  sample  variance.  From  obtaining  X",  we  can  determine  if  two  items  or 
classifications  within  PBS  A  are  dependent.  Dependency  will  provide  insight  as  to 
whether  classifications  within  PBS  A  can  be  predictive  of  each  other.  In  other  words,  if 
we  know  two  classifications  are  dependent,  by  knowing  one  of  those  factors  we  should 
have  a  clue  or  insight  as  to  what  the  other  factor  will  be.  (McClave  et.al.,  2001) 

In  order  to  detennine  dependency  across  two  classifications,  we  establish  or  start 
from  a  reference  point  that  classifications  are  independent  of  each  other.  This  will  be  our 
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null  hypothesis  (Ho).  Our  alternate  hypothesis  (Ha)  will  therefore  be  that  the  two 
classifications  are  dependent.  In  order  to  determine  if  we  accept  or  reject  the  null 
hypothesis  we  must  determine  the  X'  for  the  contingency  table,  by  using  expected  values 
(E)  for  each  cell  count.  The  following  equations  are  based  off  Table  1  as  a  sample: 

E(nn)  =  n(ri/n)(  ci/n)  =  X\C\ln 
This  is  accomplished  for  each  cell: 

E(ni2)  =  ric2/n 


E(n34)  =  r3C4/n 

Once  the  information  is  tabulated,  data  cells  that  have  an  expected  value  of  5  or  less  must 
be  removed,  as  they  will  cause  inconsistencies  within  the  table,  due  to  the  low  expected 
number.  (McClave  et.al.,  2001) 

X"  is  now  detennined  through  the  following  equation: 

X2  =  [nn  -E(nn)]2/E(nn)  +  [ni2  -  E(ni2)]2/E(n12)  +  ...  +  [n34  -  E(n34)]2/E(n34) 
In  order  to  detennine  X"  for  each  contingency  table,  the  above  equation  compares  the 
observed  and  expected  counts  in  each  cell  of  the  contingency  table.  Then  the  sum  of 
each  cell  is  taken  in  order  to  determine  X  .  Once  we  have  detennined  X  ,  we  must 
determine  our  chi-square  alpha  (X  a).  Alpha  (a)  is  the  significance  level,  usually  0.05, 
and  implies  willingness  to  accept  that  5%  (alpha)  of  the  time  a  significant  difference  will 
be  incorrectly  declared.  In  order  to  determine  X  a  ,  we  must  find  the  degrees  of  freedom 
(df),  which  is  the  shape  of  a  distribution  or  contingency  table.  To  find  degrees  of 
freedom,  we  use  the  following  equation: 
df  =  (r-l)(c-l) 
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Finally,  we  use  X  tables  to  detennine  our  X“a  based  on  our  degrees  of  freedom  (X“a  ;  df). 

In  order  to  detennine  if  we  accept  or  reject  our  null  hypothesis,  we  must  establish  a  reject 
region,  represented  like  this: 

Rejection  region:  X2  >  X2a  ;  df 

If  we  have  an  extremely  large  X2,  or  if  X2  is  larger  than  X2„  .  df,  we  reject  the  null 
hypothesis  of  independence  and  assume  the  alternate  hypothesis  is  correct  and  that  there 
is  dependence  between  classifications.  (McClave  et.al.,  2001) 

There  is  a  final  note  of  caution  that  must  be  addressed  when  using  contingency 
tables.  Contingency  tables  analysis  can  determine  if  there  is  a  dependency  between  two 
factors  or  classifications.  Contingency  tables  analysis  does  not  determine  the  extent  of 
the  relationship  between  the  two  classifications.  This  is  due  to  the  fact  that  dependence 
can  be  due  to  a  number  of  factors,  not  just  a  relationship  between  the  two  classifications. 
(McClave  et.al.,  2001) 

Validity  and  Reliability 

As  in  any  research,  there  is  a  concern  as  to  the  validity  and  reliability  of  that 
research.  Validity  can  be  defined  as  the  accuracy,  meaningfulness,  and  credibility  of  a 
model,  where  as  reliability  is  the  extent  to  which  a  measurement  instrument  yields 
consistent  results.  This  section  will  look  at  construct,  internal,  and  external  validity,  as 
well  as  the  reliability  of  the  research  being  presented.  (Leedy  and  Onnrod,  2001) 

Construct  validity  is  the  extent  to  which  an  instrument  measures  a  characteristic 
that  cannot  be  directly  observed  but  must  instead  be  inferred.  In  this  situation,  there  are 
no  inferences  by  the  DD350  system  to  collect  data  and  it  is  indifferent  to  this  aspect  by 
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strictly  focusing  on  the  collection  of  data  in  relationship  to  the  contract  action.  Inference 
of  outside  effects  to  the  data  are  minimized  by  the  use  of  regression  models,  such  as 
nominal  logistical  regression  and  contingency  tables  and  are  solely  based  on  the  data 
provided.  By  utilizing  these  methods,  the  data  can  be  viewed  in  an  unbiased  manner  and 
minimize  this  risk  of  outside  influence.  (Leedy  and  Ormrod,  2001) 

Unfortunately,  construct  validity  is  compromised  due  to  the  means  in  which  the 
data  was  gathered.  Individuals  inputting  data,  know  the  data  will  be  viewed  at  some 
point  and  can  place  bias  upon  the  data  based  on  this  perspective.  With  regards  to  this 
research,  the  data  is  already  gathered  and  there  is  no  counteraction  for  this  effect.  (Leedy 
and  Ormrod,  2001) 

The  internal  validity  is  the  extent  to  which  its  design  and  the  data  that  it  yields 
allow  the  researcher  to  draw  accurate  conclusions  about  cause-and-effect  and  other 
relationships  within  the  data.  In  order  to  minimize  this  risk,  the  research  will  use  the 
approach  of  triangulation.  Triangulation  is  when  multiple  points  of  data  are  collected  in 
the  hope  that  they  all  converge  to  support  a  particular  theory,  in  this  case,  that  there  are 
correlations  between  differing  factors  within  a  contract  action.  Nominal  logistical 
regression  will  be  used  to  find,  if  any,  these  correlations.  (Leedy  and  Ormrod,  2001) 

External  validity  is  the  extent  to  which  results  apply  to  situations  beyond  the 
study  itself.  AFMC  represents  approximately  60%  of  the  USAF  dollars  spent  and  actions 
for  service  contract  actions.  This  figure  suggests  that  AFMC  is  a  reasonable 
representation  and  sample  of  the  overall  population.  In  doing  this,  we  minimize  the  risk 
of  external  validity  in  that  any  results  from  this  research  should  reflect  a  reasonable 
assumption  with  regards  to  the  rest  of  the  USAF.  (Leedy  and  Ormrod,  2001) 
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Reliability  is  the  extent  to  which  a  measurement  instrument  or  data  yields 
consistent  results  (Leedy  and  Ormrod,  2001).  The  use  of  nominal  logistical  regression 
and  contingency  tables  in  the  research,  enhance  the  reliability.  The  results  these  analysis 
tools  provide  are  consistent  and  proven  methodologies  within  the  scientific  community 
(McClave  et.al.,  2001). 

However,  the  data  that  is  being  utilized  does  contain  issues  with  regards  to  its 
reliability.  As  previously  mentioned,  there  are  several  issues  that  plague  the  DD350 
system,  for  gathering  reliable  data.  In  addition  to  those,  there  is  a  subjective  nature 
regarding  the  DD350  system  when  inputting  data.  Several  fields  within  the  DD350, 
contain  choices  that  are  not  necessarily  identical  with  the  contract  action  being  recorded. 
This  calls  for  the  individual  inputting  the  data,  to  make  a  subjective  opinion  about  which 
field  accurately  describe  the  data  being  gathered  (GAO,  1998;  GAO;  2003).  For 
instance,  when  choosing  a  service  code,  the  individual  must  choose  the  best  code  to 
match  the  action.  This  is  a  subjective  decision  by  the  individual,  especially  if  more  then 
on  service  is  used  in  a  contract  action  and  the  code  determines  the  predominant  service 
being  performed  (GAO,  1998;  GAO;  2003). 

The  subjective  nature  of  these  inputs  can  cause  possible  discrepancies  within  the 
data  that  can  cause  outcomes  from  the  research  to  be  inaccurate  (Leedy  and  Ormrod, 
2001).  There  is  very  little  that  can  be  done  in  order  to  address  this  issue.  The  data  will 
be  taken  at  face  value  and  noted  as  a  limitation  to  this  research. 
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Summary 

Chapter  3  discusses  the  research  objective,  the  associated  research,  and 
investigative  questions  in  order  to  meet  this  objective.  The  research  design  and  the 
nature  at  which  that  data  has  been  examined  has  been  discussed.  The  means  at  which  the 
data  was  gathered,  the  data’s  validity  and  reliability,  and  the  pitfalls  to  the  data  were 
addressed.  Finally,  a  discussion  of  the  methodologies  that  will  be  used  to  analyze  this 
data  was  discussed.  Chapter  4  will  address  the  data  analysis,  findings  based  from  this 
analysis,  and  any  conclusions  that  can  be  drawn  from  the  research  accomplished. 
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IV.  Results  and  Analysis 


This  chapter  provides  a  summary  of  the  results  of  the  research.  The  research 
results  are  represented  in  three  sections.  The  first  section  includes  demographic  results 
and  addresses  investigative  question  1 .  The  second  section  includes  nominal  logistical 
regression  analysis  and  conclusions  based  on  investigative  question  2.  The  third  section 
addresses  contingency  tables  and  conclusions  based  on  investigative  question  2. 
Appendix  1  provides  complete  demographic  breakouts  that  are  not  supplied  in  this 
chapter.  Appendix  2  and  3  provide  supplemental  findings  of  nominal  logistical 
regression  and  contingency  tables  not  provided  in  this  chapter. 

Data  Analysis 

Research  Question.  Based  on  the  requirements  placed  on  Perfonnance  Based 
Service  Acquisition  (PBSA)  and  the  limited  research  to  date,  this  thesis  will  serve  as  an 
exploratory  study  in  which  future  research  can  be  based.  The  research  question  of  this 
thesis  is:  What  is  the  status  of  PBSA  implementation  in  the  US  Air  Force  (USAF)?  The 
following  investigative  questions  were  developed  in  order  to  answer  the  research 
question: 

1 .  Are  the  goals,  as  established  by  Dr.  Gansler  and  National  Defense  Authorization 
Act  (NDAA),  being  met  by  Air  Force  Material  Command  (AFMC)  and  the  USAF 
and  are  there  any  trends  to  suggest  future  usage  of  PBSA? 
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2.  What  factors,  if  any,  are  related  to  whether  a  contract  is  classified  as  having  been 


awarded  using  PBSA  strategies? 


Demographics.  In  order  to  answer  investigative  question  1,  demographics  were 
utilized  to  gain  a  basic  insight  into  the  usage  of  PBSA.  A  summarized  version  of  these 
demographics  can  be  found  in  Tables  2,  3,  and  4. 


Table  2.  Summarized  Demographics  for  AFMC  FY  2002 


AFMC  FY  2002 

Total  Number  of 
Actions  = 

9906 

Actions  PBSA  = 

2490 

%  of  Total  Actions 
that  are  PBSA  = 

25.04% 

Total  Dollars  = 

$7,044,436,993.00 

Total  PBSA  $  = 

$2,213,182,268.00 

Total  %$  = 

31.42% 

Table  3.  Summarized  Demographics  for  AFMC  FY  2003 


AFMC  FY  2003 

Total  Number  of 
Actions  = 

20307 

Actions  PBSA  = 

2964 

%  of  Total  Actions 
that  are  PBSA  = 

14.60% 

Total  Dollars  = 

$16,459,023,441.00 

Total  PBSA  $  = 

$2,834,679,428.00 

Total  %$  = 

17.22% 
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Table  4.  Summarized  Demographics  for  USAF  FY  2003 


USAF  FY  2003 

Total  Number  of 
Actions  = 

35607 

Actions  PBSA  = 

10191 

%  of  Total  Actions 
that  are  PBSA  = 

28.62% 

Total  Dollars  = 

$27,911,859,775.00 

Total  PBSA  $  = 

$8,761,877,311.00 

Total  %$  = 

31.39% 

Looking  first  at  the  AFMC  data  for  FY  2002  and  FY  2003,  we  can  see  that  the 
actions  for  services,  as  well  as  the  dollars,  have  more  then  doubled  between  the  two 
years.  While  AFMC  has  more  then  doubled  its  service  acquisitions  between  the  two 
years,  the  percentage  of  actions  and  dollars  being  utilized  under  PBSA  has  decreased. 
Between  FY  2002  and  FY  2003,  the  number  of  action  AFMC  has  awarded  under  PBSA 
have  only  increased  by  474,  a  19%  increase,  where  as  the  overall  number  of  service 
actions  for  AFMC  has  increased  104%.  Dollars  for  PBSA  increased  28%  from  FY  2002 
to  FY  2003,  while  overall  service  dollars  for  AFMC  increased  133%. 

From  this  initial  look  across  two  years,  the  results  for  AFMC  are  discouraging  in 
2003.  AFMC  represents  approximately  60%  of  the  service  actions  and  dollar 
expenditures  for  the  USAF.  The  NDAA  for  FY  2002  required  that  25%  of  service 
actions  be  PBSA  for  FY  2003  (NDAA,  2002).  AFMC  FY  2003  fell  short  of  reaching  this 
goal,  by  10.4%  of  its  actions  and  7.78%  of  its  dollars  spent.  It  should  be  noted  that  there 
is  an  outlier  in  the  data  for  one  contract  action  with  the  value  of  $  1 .75B.  If  this  outlier  is 


33 


removed,  AFMC  would  increase  its  PBSA  usage  to  19.5%.  This  still  falls  short  of 
established  goals. 

When  looking  at  the  USAF  as  a  whole,  a  snap  shot  of  current  time  looks 
promising.  The  USAF  showed  28.62%  of  all  its  service  acquisitions  are  PBSA  and  that 
3 1 .39%  of  its  dollars  spent  on  services  are  PBSA.  This  meets  the  required  25%  for  FY 
2003.  From  this,  two  critical  points  can  be  ascertained.  First,  the  USAF  is  meeting  and 
exceeding  the  requirements  lad  out  for  PBSA.  Second,  if  AFMC  represents  60%  of 
service  actions/dollars  spent  and  is  falling  short  of  the  established  requirements,  then  the 
remainder  of  the  USAF  must  be  exceeding  the  requirements.  To  be  more  precise,  the 
remaining  40%  of  the  USAF  is  awarding  service  actions  at  47.24%  and  service  dollars  at 
a  5 1 .75%.  This  is  a  significant  amount  and  comes  close  to  meeting  the  NDAA 
requirement  for  FY  2005  of  50%  services  must  be  PBSA  (NDAA,  2002). 

When  looking  further  into  the  data  and  reviewing  Chapter  2,  it  became  apparent 
that  there  were  three  probable  indicators  of  PBSA:  service  code  (type  of  service),  contract 
type,  and  type  of  entity  (business  size).  Table  5,  6,  and  7  show  a  summary  and  key 
highlighted  areas  for  service  codes. 
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Table  5.  Summary  of  Service  Code  for  AFMC  FY  2002 


AFMC  FY  02  Separated  PBSA  by  Service  Code 

Service 

Code 

Service  Code  Description 

%  of  Actions 
Yes  PBSA 
by  Service 
Code 

%  of 

Dollars  Yes 
PBSA  by 
Service 
Code 

%  of  Service 
Code  Dollars 
by  Total 
Service  Code 
Dollars 

AZ16 

RDTE/Other  Research  & 
Development-Mqmt  Support 

32.52% 

90.20% 

3.97% 

B599 

Other  Special  Studies  & 
Analyses 

25.48% 

12.28% 

5.60% 

D306 

ADP  Systems  Analysis  Services 

18.93% 

25.19% 

1 .25% 

D307 

Other  ADP  & 

Telecommunication  Services 

1 .33% 

0.03% 

1.19% 

D399 

Other  ADP  & 

Telecommunication  Services 

37.46% 

47.95% 

1 .89% 

J015 

Maint  &  Repair  of  Eq/Aircraft 
Structural  Comps 

6.45% 

8.85% 

6.06% 

J016 

Maint  &  Repair  of  Eq/Aircraft 
Comps  &  Accys 

7.50% 

1 .49% 

1 .99% 

J058 

Maint  &  Repair  of 
Eq/Communication  Equipment 

16.13% 

4.87% 

1.10% 

J069 

Maint  &  Repair  of  Eq/T rain ing 
Aids  &  Devices 

67.23% 

51.84% 

2.42% 

J070 

Maint  &  Repair  of  Eq/ADP 
Equip  &  Supplies 

1 .65% 

0.82% 

4.10% 

J099 

Maint  Si  Repair  of 
Eq/Miscellaneous  Equipment 

41.01% 

31.21% 

2.03% 

KOI  5 

Modification  of  Eq/Aircraft 
Structural  Comps 

9.76% 

5.05% 

1 .64% 

KOI  6 

Modification  of  Eq/Aircraft 
Comps  Si  Accys 

0.00% 

0.00% 

1 .33% 

K058 

Modification  of 

Eq/Communication  Equipment 

0.00% 

0.00% 

1 .05% 

K069 

Modification  of  Eq/T  rain  ing 
Aids  Si  Devices 

72.97% 

54.15% 

1.15% 

L014 

Tech  Rep  Svcs/Guided  Missiles 

71.14% 

64.02% 

12.67% 

R414 

Systems  Engineering  Services 

23.36% 

14.07% 

18.62% 

R425 

Engineering  Technical  Services 

34.77% 

39.85% 

7.62% 

R706 

Logistics  Support  Services 

52.34% 

86.40% 

4.01% 

Totals: 

25.58% 

32.22% 

79.69% 
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Table  6.  Summary  of  Service  Code  for  AFMC  FY  2003 


AFMC  FY  03  Separated  PBSA  by  Service  Code 

Service 

Code 

Service  Code  Description 

%  of  Actions 
Yes  PBSA  by 
Service  Code 

%  of  Dollars 
Yes  PBSA 
by  Service 
Code 

%  of  Service 
Code  Dollars  by 
Total  Service 
Code  Dollars 

AC14 

RDTE/Aircraft-Demo/Valid 

0.00% 

0.00% 

1 .67% 

AC15 

RDTE/Aircraft-Eng/Manuf 

Develop 

0.00% 

0.00% 

6.36% 

AC53 

RDTE/Weapons-Adv  Tech  Dev 

0.00% 

0.00% 

3.65% 

AC63 

RDTE/Electronics  & 
Communication  Eq-AdvTech 
Dev 

0.00% 

0.00% 

1.13% 

AC65 

RDTE/Electronics  & 
Communication  Eq-Eng/Manuf 
Dev 

0.00% 

0.00% 

2.53% 

AD92 

RDTE/Other  Defense-Applied 
Research 

0.17% 

0.05% 

2.49% 

AD93 

RDTE/Other  Defense-Adv  Tech 
Dev 

0.00% 

0.00% 

1 .83% 

AZ13 

RDTE/Other  Research  & 
Development-Adv  Tech  Dev 

0.00% 

0.00% 

3.53% 

AZ14 

RDTE/Other  Research  & 
Development- Demo/Valid 

0.00% 

0.00% 

1 .26% 

AZ16 

RDTE/Other  Research  & 
Development-Mgmt  Support 

35.88% 

92.61% 

1.72% 

B599 

Other  Special  Studies  & 
Analyses 

47.20% 

27.56% 

3.15% 

H216 

Equip  &  Mats  Testing/Aircraft 
Comps  Si  Accys 

0.00% 

0.00% 

10.95% 

J015 

Maint  Si  Repair  of  Eq/Aircraft 
Structural  Comps 

9.15% 

5.73% 

3.07% 

J016 

Maint  Si  Repair  of  Eq/Aircraft 
Comps  Si  Accys 

3.85% 

0.23% 

2.37% 

J069 

Maint  Si  Repair  of  Eq/T raining 
Aids  Si  Devices 

82.79% 

88.06% 

1.41% 

J070 

Maint  Si  Repair  of  Eq/ADP 
Equip  Si  Supplies 

3.20% 

0.84% 

2.62% 

L014 

Tech  Rep  Svcs/Guided  Missiles 

60.12% 

70.47% 

5.61% 

R414 

Systems  Engineering  Services 

25.59% 

21.09% 

8.38% 

R425 

Engineering  Technical  Services 

42.60% 

31.25% 

2.76% 

R499 

Other  Professional  Services 

24.76% 

90.30% 

2.27% 

Totals: 

14.64% 

17.40% 

76.61% 

36 


Table  7.  Summary  of  Service  Code  for  USAF  FY  2003 


USAF  FY  03  Separated  PBSA  by  Service  Code 

Service 

Code 

Service  Code  Description 

%  of  Actions 
Yes  PBSA  by 
Service  Code 

%  of  Dollars 
Yes  PBSA 
by  Service 
Code 

%  of  Service 
Code  Dollars  by 
Total  Service 
Code  Dollars 

AC15 

RDTE/Aircraft-Eng/Manuf  Develop 

0.00% 

0.00% 

3.78% 

AC22 

RDTE/Missile  and  Space  Systems- 
Applied  Research 

0.00% 

0.00% 

3.15% 

AC23 

RDTE/Missile  and  Space  Systems-Adv 
Tech  Dev 

2.27% 

0.57% 

3.46% 

AC25 

RDTE/Missile  and  Space  Systems- 
Enq/Manuf  Devel 

0.00% 

0.00% 

2.10% 

AC53 

RDTE/Weapons-Adv  Tech  Dev 

0.00% 

0.00% 

2.17% 

AC65 

RDTE/Electronics  &  Communication  Eq- 
Enq/Manuf  Dev 

0.00% 

0.00% 

1 .85% 

AD92 

RDTE/Other  Defense-Applied  Research 

0.90% 

1.10% 

1 .50% 

AD93 

RDTE/Other  Defense-Adv  Tech  Dev 

0.00% 

0.00% 

1 .09% 

AR25 

RDTE/Space-Science  Applications- 
Enq/Manuf  Devel 

66.67% 

69.86% 

2.01% 

AR55 

RDTE/Sp  &  Terrestrial  Application- 
Enq/Manuf  Devel 

0.00% 

0.00% 

1 .54% 

AZ13 

RDTE/Other  Research  &  Development- 
Adv  Tech  Dev 

3.97% 

0.13% 

2.10% 

AZ16 

RDTE/Other  Research  &  Development- 
Mqmt  Support 

25.68% 

83.73% 

1.13% 

B599 

Other  Special  Studies  &  Analyses 

48.99% 

27.82% 

1 .88% 

H216 

Equip  &  Mats  Testing/Aircraft  Comps  Si 
Accys 

0.00% 

0.00% 

6.51% 

J015 

Maint  Si  Repair  of  Eq/Aircraft  Structural 
Comps 

18.93% 

26.04% 

2.73% 

J016 

Maint  Si  Repair  of  Eq/Aircraft  Comps  Si 
Accys 

6.32% 

0.58% 

1.41% 

J070 

Maint  Si  Repair  of  Eq/ADP  Equip  Si 
Supplies 

5.42% 

1 .42% 

1 .92% 

L014 

Tech  Rep  Svcs/Guided  Missiles 

60.12% 

70.47% 

3.34% 

R414 

Systems  Engineering  Services 

31.46% 

26.10% 

5.33% 

R425 

Engineering  Technical  Services 

47.41% 

35.04% 

2.23% 

R499 

Other  Professional  Services 

30.24% 

72.94% 

1.79% 

S216 

Facilities  Operations  Support  Services 

72.41% 

77.31% 

3.25% 

V121 

Air  Charter  for  Things 

99.78% 

98.86% 

6.00% 

V221 

Passenger  Air  Charter  Service 

64.66% 

93.96% 

3.88% 

Totals: 

27.66% 

31.37% 

66.15% 
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The  service  codes  above  represent  1%  of  the  service  dollars  spent  or  greater,  or 
$60M  or  greater,  and  provide  a  high  level  view  of  the  data.  When  analyzing  these 
results,  we  can  see  some  common  service  codes,  such  as  AZ16  (Other  Research  and 
Development  Management  Support)  where  the  expenditure  for  this  service  is  greater  then 
$300M  and  greater  then  80%  for  the  USAF  is  being  accomplished  as  PBSA.  AFMC,  for 
service  AZ16,  is  greater  then  90%  for  PBSA  utilization.  By  viewing  this  data,  we  can 
find  common  points  that  are  being  greatly  utilized,  represent  the  greatest  point  of 
expenditure,  and  try  to  apply  them  to  other  services. 

Similar  to  finding  the  strong  points  in  the  data,  we  can  do  the  same  for  the  weak 
points  of  the  data.  For  example,  R414  (Systems  Engineering  Services)  which  in  both  the 
AFMC  and  USAF  represents  5%  or  more  of  service  dollars  spent,  but  this  service  is  only 
utilized  to  approximately  25%  of  PBSA.  Another  example  would  be  service  code  H216, 
Equipment  and  Maintenance  Testing  of  Aircraft  Components.  H216  represents  $1.7B, 
6.5%,  of  the  USAF  budget  for  services,  with  none  of  the  requirements  utilized  under 
PBSA.  Key  areas,  such  as  these,  need  to  be  looked  at  in  further  depth  in  order  to 
determine  if  they  should  be  exempt  from  PBSA  usage  or  if  they  need  to  be  pursued  in 
greater  depth  for  PBSA  utilization.  It  is  through  identification  of  these  key  areas,  that  the 
USAF  will  make  its  greatest  strides  in  obtaining  its  goals. 

The  second  key  indicator,  contract  type,  is  summarized  in  Tables  8,  9,  and  10. 
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Table  8.  Summary  of  Contract  Type  for  AFMC  FY  2002 


AFMC  FY  02  Se\ 

parated  PBSA  by  Contract  Type 

Contract 

Type 

Contract  Type 
Description 

%  of  Actions 
Yes  PBSA  by 
Contract  Type 

%  of 
Dollars 
Yes  PBSA 
by 

Contract 

Type 

%  of  Contract 
Type  Dollars 
by  Total 
Contract  Type 
Dollars 

A 

Fixed  Price 
Redetermination 

33.33% 

63.49% 

0.02% 

J 

Firm  Fixed  Price 

26.58% 

34.30% 

34.93% 

K 

Fixed  Price 
Economic  Price 
Adjustment 

50.00% 

74.58% 

0.03% 

L 

Fixed  Price 
incentive 

13.33% 

77.60% 

4.28% 

M 

Fixed  Price 
Award  Fee 

72.06% 

57.63% 

0.92% 

R 

Cost  Plus 
Award  Fee 

47.20% 

47.51% 

21.12% 

S 

Cost  Contract 

16.39% 

29.05% 

1 .04% 

T 

Cost  Sharing 

28.57% 

19.09% 

0.02% 

U 

Cost  Plus  Fixed 
Fee 

9.37% 

18.44% 

10.86% 

V 

Cost  Plus 
Incentive  Fee 

9.28% 

2.49% 

3.61% 

Y 

Time  and 
Materials 

28.01% 

15.70% 

21.14% 

Z 

Labor  Hour 

32.62% 

29.65% 

1 .97% 

Not  Listed 

18.18% 

32.29% 

0.05% 

Totals: 

25.58% 

32.22% 

100.00% 
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Table  9.  Summary  of  Contract  Type  for  AFMC  FY  2003 


AFMC  FY  03  Se| 

parated  PBSA  by  Contract  Type 

Contract 

Type 

Contract  Type 
Description 

%  of  Actions 
Yes  PBSA  by 
Contract  Type 

%  of 
Dollars 
Yes  PBSA 
by 

Contract 

Type 

%  of  Contract 
Type  Dollars 
by  Total 
Contract  Type 
Dollars 

A 

Fixed  Price 
Redetermination 

50.00% 

15.84% 

0.00% 

J 

Firm  Fixed  Price 

20.30% 

16.55% 

32.20% 

K 

Fixed  Price 
Economic  Price 
Adjustment 

83.33% 

99.51% 

0.03% 

L 

Fixed  Price 
incentive 

21.74% 

37.36% 

4.26% 

M 

Fixed  Price 
Award  Fee 

38.89% 

43.74% 

0.74% 

R 

Cost  Plus 
Award  Fee 

28.18% 

28.49% 

23.17% 

S 

Cost  Contract 

12.06% 

5.16% 

5.70% 

T 

Cost  Sharing 

7.27% 

1 .94% 

0.13% 

U 

Cost  Plus  Fixed 
Fee 

1 .30% 

3.40% 

16.75% 

V 

Cost  Plus 
Incentive  Fee 

38.16% 

12.75% 

3.54% 

Y 

Time  and 
Materials 

21.90% 

16.11% 

12.96% 

Z 

Labor  Hour 

22.57% 

21.50% 

0.49% 

Not  Listed 

42.86% 

83.07% 

0.02% 

Totals: 

14.64% 

17.40% 

100.00% 
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Table  10.  Summary  of  Contract  Type  for  USAF  FY  2003 


USAF  FY  03  Se| 

parated  PBSA  by  Contract  Type 

Contract 

Type 

Contract  Type 
Description 

%  of  Actions 
Yes  PBSA  by 
Contract  Type 

%  of 
Dollars 
Yes  PBSA 
by 

Contract 

Type 

%  of  Contract 
Type  Dollars 
by  Total 
Contract  Type 
Dollars 

A 

Fixed  Price 
Redetermination 

33.33% 

2.10% 

0.02% 

J 

Firm  Fixed  Price 

36.37% 

44.14% 

37.69% 

K 

Fixed  Price 
Economic  Price 
Adjustment 

84.03% 

82.07% 

0.30% 

L 

Fixed  Price 
incentive 

32.74% 

42.95% 

3.80% 

M 

Fixed  Price 
Award  Fee 

58.30% 

68.23% 

1 .06% 

R 

Cost  Plus 
Award  Fee 

39.98% 

31.24% 

28.69% 

S 

Cost  Contract 

12.11% 

5.88% 

3.83% 

T 

Cost  Sharing 

7.27% 

1 .94% 

0.08% 

U 

Cost  Plus  Fixed 
Fee 

3.45% 

6.00% 

11.17% 

V 

Cost  Plus 
Incentive  Fee 

48.92% 

13.48% 

4.91% 

Y 

Time  and 
Materials 

24.27% 

18.43% 

8.01% 

Z 

Labor  Hour 

27.60% 

32.78% 

0.35% 

Not  Listed 

10.20% 

20.19% 

0.09% 

Totals: 

27.66% 

31.37% 

100.00% 

41 


Like  that  of  service  codes,  we  can  identify  key  areas  by  contract  type  that  utilize 
the  most  expenditure  for  services.  Looking  at  Tables  8,  9,  and  10,  we  can  identify  four 
different  contract  types  with  the  highest  expenditures  for  total  service  acquisitions:  J 
(Firm  Fixed  Price  (FFP)),  R  (Cost  Plus  Award  Fee),  U  (Cost  Plus  Fixed  Fee),  and  Y 
(Time  and  Materials).  Of  particular  interest  and  specifically  pointed  out  as  being  utilized 
for  PBSA  is  the  FFP  contract  type.  In  FY  2003,  the  USAF  utilized  44%  of  FFP  as  PBSA 
and  AFMC  utilized  even  lower.  These  four  categories  represent  85%  of  the  dollars  spent 
on  services.  It  is  areas  such  as  these  that  improvement  can  be  obtained  in  order  to  reach 
established  PBSA  requirements. 

The  final  area  within  demographics  that  the  research  will  touch  upon  is  that  of 
service  actions/dollars  spent  for  services  against  that  of  business  size.  Tables  11,  12,  and 
13  represent  a  summary  of  these  findings. 
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Table  11.  Summary  of  Business  Size  for  AFMC  FY  2002 


AFMC  FY  02  Separated  PBSA  by  Business  Size 

Business 

Size 

Business  Size 
Description 

%  of  Actions 
Yes  PBSA 
by  Business 
Size 

%  of 
Dollars 
Yes  PBSA 
by 

Business 

Size 

%  of  Business 
Size  Dollars 
by  Total 
Business  Size 
Dollars 

A 

Small 

Disadvantaged 
Business  (SDB) 
Performing  in  the 
US 

36.36% 

36.81% 

5.61% 

B 

Other  Small 
Business 

29.75% 

36.48% 

8.88% 

C 

Large  Business 
Performing  in  the 
US 

21.13% 

32.25% 

80.43% 

D 

JWOD  Participating 
Nonprofit  Agency 

47.83% 

60.28% 

0.38% 

F 

Hospital 

100.00% 

100.00% 

0.01% 

L 

Foreign  Concern  or 
Entity 

13.95% 

10.24% 

0.26% 

M 

Domestic  Firm 
Performing  Outside 
the  US 

24.62% 

10.27% 

2.51% 

T 

Historical  Black 
College  or 
University  (HBCU) 

0.00% 

0.00% 

0.00% 

U 

Minority  Institution 
(Ml) 

0.00% 

0.00% 

0.00% 

V 

Other  Educational 

24.66% 

39.73% 

0.71% 

z 

Other  Nonprofit 

40.35% 

14.71% 

1 .22% 

Not  Listed 

0.00% 

0.00% 

0.00% 

Totals: 

25.58% 

32.22% 

100.00% 
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Table  12.  Summary  of  Business  Size  for  AFMC  FY  2003 


AFMC  FY  03  Separated  PBSA  by  Business  Size 

Business 

Size 

Business  Size 
Description 

%  of  Actions 
Yes  PBSA 
by  Business 
Size 

%  of 
Dollars 
Yes  PBSA 
by 

Business 

Size 

%  of  Business 
Size  Dollars 
by  Total 
Business  Size 
Dollars 

A 

Small 

Disadvantaged 
Business  (SDB) 
Performing  in  the 
US 

30.45% 

34.37% 

3.40% 

B 

Other  Small 
Business 

13.17% 

20.02% 

8.35% 

C 

Large  Business 
Performing  in  the 
US 

12.96% 

17.57% 

78.58% 

D 

JWOD  Participating 
Nonprofit  Agency 

67.74% 

72.39% 

0.22% 

F 

Hospital 

0.00% 

0.00% 

0.00% 

L 

Foreign  Concern  or 
Entity 

16.84% 

9.43% 

0.36% 

M 

Domestic  Firm 
Performing  Outside 
the  US 

8.94% 

9.50% 

2.50% 

T 

Historical  Black 
College  or 
University  (HBCU) 

0.00% 

0.00% 

0.04% 

U 

Minority  Institution 
(Ml) 

0.00% 

0.00% 

0.15% 

V 

Other  Educational 

8.55% 

4.87% 

4.50% 

z 

Other  Nonprofit 

16.23% 

4.99% 

1 .89% 

Not  Listed 

42.86% 

83.07% 

0.02% 

Totals: 

14.64% 

17.40% 

100.00% 
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Table  13.  Summary  of  Business  Size  for  USAF  FY  2003 


USAF  FY  03  Separated  PBSA  by  Business  Size 

Business 

Size 

Business  Size 
Description 

%  of  Actions 
Yes  PBSA 
by  Business 
Size 

%  of 
Dollars 
Yes  PBSA 
by 

Business 

Size 

%  of  Business 
Size  Dollars 
by  Total 
Business  Size 
Dollars 

A 

Small 

Disadvantaged 
Business  (SDB) 
Performing  in  the 
US 

35.08% 

49.27% 

4.39% 

B 

Other  Small 
Business 

23.31% 

31.90% 

7.42% 

C 

Large  Business 
Performing  in  the 
US 

26.33% 

28.62% 

74.70% 

D 

JWOD  Participating 
Nonprofit  Agency 

68.69% 

60.65% 

0.68% 

F 

Hospital 

0.00% 

0.00% 

0.00% 

L 

Foreign  Concern  or 
Entity 

46.88% 

42.88% 

1.51% 

M 

Domestic  Firm 
Performing  Outside 
the  US 

45.22% 

58.34% 

4.63% 

T 

Historical  Black 
College  or 
University  (HBCU) 

4.17% 

3.49% 

0.03% 

U 

Minority  Institution 
(Ml) 

7.27% 

1 .40% 

0.09% 

V 

Other  Educational 

7.94% 

5.58% 

2.83% 

z 

Other  Nonprofit 

14.73% 

41.93% 

3.62% 

Not  Listed 

10.20% 

20.19% 

0.09% 

Totals: 

27.66% 

31.37% 

100.00% 
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Unlike  previous  key  indicators,  business  size  breaks  down  to  almost  one  key 
element,  large  business  (code  C  above)  holds  the  predominance  of  the  business  for  the 
USAF  in  services.  Based  on  the  data,  75%  of  the  dollars  spent  of  services  goes  to  large 
business,  yet  only  28.6%  of  those  dollars  were  awarded  towards  PBSA.  In  order  for  any 
improvement  to  occur,  the  USAF  must  rely  on  business  practices  that  come  from  the 
commercial  community  and  large  business  represents  a  great  portion  of  this  industry 
(Moore  et.al.,  2002).  If  we  look  at  industries  best  practices,  there  is  a  possibility  to 
determine  how  the  commercial  sector  is  approaching  services,  convert  additional  dollars 
and  actions  to  PBSA  (Moore  et.al.,  2002).  Now  that  demographics  has  identified  three 
key  indicators  within  PBSA,  nominal  logistical  regression  will  identify  any  predictive 
relationships  with  the  data. 

Nominal  Logistical  Regression.  As  stated  in  Chapter  3,  nominal  logistical 
regression  utilized  nominal  data  or  data  that  is  verbal  in  nature  and  can  be  converted  into 
a  numerical  value.  The  results  from  these  tests  were  inconclusive.  The  highest  R  that 
could  be  found  using  a  single  independent  variable  was  that  of .  1 .  This  means  that  none 
of  the  data  when  looked  at  individually  against  the  dependent  variable  of  PBSA, 
produced  a  very  low  explanatory  for  the  prediction  of  whether  a  contract  would  be  PBSA 
or  not  (McClave  et.al.,  2001).  It  is  important  to  note  that  many  of  the  variables  used, 
including  that  of  service  codes,  were  classified  as  unstable,  meaning  that  a  regression 
model  could  not  be  established.  When  multiple  independent  variables  were  utilized, 
including  contract  type  and  business  size,  the  greatest  R  that  could  be  achieved  was  .  145 
and  required  13.5%  of  the  data  to  be  removed  in  order  the  model  to  have  any  predictive 
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capability.  The  justification  for  removing  data  points  is  based  on  data  that  was  either 
unduly  influential  on  the  model  or  was  considered  a  major  outlier  with  regards  to  the  rest 
of  the  data  (McClave  et.al.,  2001).  Table  14  summarizes  the  results  against  the 
dependant  variable,  PBSA. 


Table  14.  Summary  of  Nominal  Logistical  Regression 


Variable 

R“  (U) 

Prob>ChiSq 

B8,  Dollars 

0.0000 

0.4533 

B9,  Foreign  Military  Sales 

0.0010 

<.0001 

B13A,  Contract  or  Order 

0.0041 

<.0001 

B13B,  Type  if  ID  Contract 

0.0082 

<.0001 

B13C  Single  or  Multiple  Award 

ID  Contract 

0.0019 

<.0001 

B13D,  Modification 

0.0165 

<.0001 

B15,  IT  Product  or  Service 

0.0760 

<.0001 

Cl,  Synopsis 

0.0012 

<.0001 

C3,  Extent  Competed 

0.0154 

<.0001 

C5,  Type  of  Contract 

0.1003 

0.0000 

C14,  Commercial  Item 

0.0295 

<.0001 

C5,  Type  of  Contract;  C14, 
Commercial  Item;  D1A,  Type  of 
Entity 

0.1454 

0.0000 

Con  tingency  Tables.  Despite  the  apparent  failure  of  the  nominal  logistical 
regression,  contingency  tables  identified  a  different  point  of  view  when  looking  at  key 
variables  within  a  contract.  Contingency  tables  were  run  on  all  variables  that  were 
considered  stable  during  nominal  logistical  regression.  Since  this  was  a  secondary  step  in 
determining  correlations  between  variables,  all  unstable  variables  were  considered  to 
have  little  value  in  the  process  and  the  focus  remained  on  those  variables  that  showed 
some  predictive  value  to  the  dependent  variable.  The  only  exceptions  to  this  were  the 
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contingency  tables  for  service  code  and  business  size.  Despite  their  unstable  results  in 
nominal  logistical  regression,  these  two  variables  has  been  identified  as  key  elements 
within  PBSA.  The  results  that  will  be  obtained  from  using  these  two  variables,  provides 
additional  value  to  the  process  in  determining  if  they  are  truly  key  elements.  A 
summarized  listing  of  results  for  independent  variables  against  the  dependent  variable 
PBSA,  can  be  seen  in  Table  15. 


Table  15.  Summary  of  Contingency  Tables 


Independent  Variable 

X? 

x2a 

Degrees  of 
Freedom  (df) 

Result 

B12A,  Service  Code 

7864.938 

132.157 

103 

Dependent 

B13A  Contract 

93.893 

12.5916 

6 

Dependent 

B13B  Type  ID  Contract 

150.697 

5.99147 

2 

Dependent 

B13C  Multi  or  Single  ID 
Contract 

28.153 

3.84146 

1 

Dependent 

B13D  Modification 

282.445 

12.5916 

6 

Dependent 

B15  IT  Products  or 
Service 

453.387 

11.0705 

5 

Dependent 

Cl  Synopsis 

29.693 

5.99147 

2 

Dependent 

C3  Extent  Competed 

457.758 

7.81473 

3 

Dependent 

C5  Type  of  Contract 

2017.582 

16.9190 

9 

Dependent 

C8  Solicitation 

936.568 

12.5916 

6 

Dependent 

CIO  Labor  Standards 

2243.456 

7.81473 

3 

Dependent 

C14  Commercial  Item 

871.347 

3.84146 

1 

Dependent 

D1A  Type  of  Entity 
(Small,  DSB,  Large,  etc.) 

677.967 

15.5073 

8 

Dependent 
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The  contingency  table  of  B12A  (Service  Code)  required  the  removal  of  15%  of 
the  data  and  narrowed  the  number  of  service  codes  utilized  down  to  104.  The 
justification  for  removing  the  data,  as  explained  in  Chapter  3,  is  that  once  the  infonnation 
is  tabulated,  data  cells  that  have  an  expected  value  of  5  or  less  must  be  removed,  as  they 
will  cause  inconsistencies  within  the  table,  due  to  the  low  expected  number  (McClave 
et.al.,  2001).  With  the  remaining  service  codes,  it  was  determined  that  dependence 
between  the  two  variables,  service  codes  and  whether  a  contract  is  PBS  A  or  not,  does 
exist.  This  tells  us  that  if  we  know  one  of  the  104  service  codes,  we  have  some  insight  as 
to  PBSA.  The  extent  of  this  dependency  is  not  known  from  this  table;  this  would  have  to 
be  identified  through  regression,  but  simple  that  it  does  exist.  It  goes  to  reason,  that  if  we 
know  a  contract  will  have  that  particular  service  code,  we  will  have  some  idea  as  to 
whether  an  action  is  PBSA  or  not.  It  is  important  to  know  that  a  relationship  does  exist 
between  the  variables  and  that  the  events  are  connected,  but  there  can  be  possible  issues 
in  the  way  of  anomalies  in  the  data  that  must  be  addressed.  (McClave  et.al.,  2001) 

When  looking  at  the  contingency  table  for  B 12A,  service  codes,  it  becomes 
apparent  that  in  some  of  the  cells,  such  as  AC  12  (Applied  Research  for  Aircraft),  there  is 
an  actual  count  of  0,  with  an  expected  count  of  much  greater  value.  Contingency  tables 
will  calculate  all  cells  in  the  same  fashion,  with  expected  counts  of  5  or  greater  for  all 
cells  in  the  table.  We  know  from  research,  that  research  and  development  has  historically 
not  been  utilized  under  PBSA  (FAR  Part  35).  This  explains  why  the  actual  count  is  zero 
and  yet  there  is  an  expected  count  in  the  cell.  Additional  anomalies,  such  as  those  with 
higher  actual  counts  then  those  of  expected,  such  as  AZ16  (Research  and  Development 
Management  Support),  are  also  expected.  Like  that  of  cells  with  lower  actual  counts  then 
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expected  ones,  higher  actual  counts  contend  with  the  same  issue  in  reverse.  Based  on 
contingency  table  cell  formulas  and  that  we  know  90%  of  the  AZ16  service  is  utilized 
under  PBS  A,  it  is  anticipated  that  this  actual  count  will  be  higher  then  the  expected  count 
displayed.  (McClave  et.al.,  2001) 

The  next  batch  of  codes  that  were  looked  at  included  the  B 13  (Kind  of  Action) 
series:  B13A  (Contract  or  Order),  B13B  (Type  of  Independent  Delivery  Contract),  B13C 
(Multi  or  Single  Independent  Delivery  Contract),  and  B13D  (Modification).  All  four 
variables  showed  dependence  against  PBSA.  This  result  is  again  expected  despite  the 
results  from  nominal  logistical  regression.  If  we  know  whether  the  action  being  taken  is 
a  contract  or  order  against  an  existing  contract,  then  we  would  have  an  insight  as  to 
whether  an  item  is  PBSA.  Modification  and  type  of  modification  is  slightly  different  in 
that  if  we  know  that  a  contract  or  order  is  already  PBSA  or  not,  that  the  modification  is 
coded  in  the  same  respect  and  would  naturally  have  dependence  on  the  predecessor  work. 

When  looking  at  the  variable  B 15  (Information  Technology  Product  or  Service)  it 
becomes  vary  apparent  that  two  codes,  those  listing  commercial  or  non-commercial 
service,  are  the  primary  drivers  establishing  dependence.  This  will  coincide  with  the 
dependence  found  in  C14  (Commercial  Item)  and  whether  an  item  is  commercial,  yes  or 
no.  The  next  two  table  Cl  (Synopsis)  and  C3  (Extent  Competed)  both  show  dependency 
and  if  we  know  this  information,  like  before,  can  give  us  insight  to  PBSA. 

C5  (Contract  Type)  is  one  of  the  few  variables  that  exhibited  signs  during 
nominal  logistic  regression  to  have  some  glimmer  of  hope  as  to  having  a  predictive 
capability.  If  we  know  the  contract  type,  we  should  have  a  reasonable  idea  to  that  of  a 
contract  action  being  PBSA  or  not.  We  know  from  demographics  that  FFP  type  contracts 
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exhibit  44%  of  the  USAF’s  contract  actions  to  be  PBSA.  However,  there  was  an 
anomaly  identified  for  type  U,  Cost  Plus  Fixed  Fee  (CPFF),  contracts  with  a  lower  actual 
then  expected  count.  CPFF  is  typically  used  for  research  and  development  and  since  we 
know  from  the  past  that  many  of  these  type  efforts  were  not  awarded  under  PBSA,  it  goes 
to  reason  that  the  type  of  contract  used  in  this  area  would  follow  suit. 

The  next  two  tables,  C8  (Solicitation  Procedures)  and  CIO  (Labor  Standards)  are 
typically  more  products  of  previous  factors.  In  other  words,  if  we  know  what  type  of 
contract  type  we  are  going  to  use,  if  know  what  type  of  service  code  it  will  fall  under, 
then  these  are  drivers  as  to  what  solicitation  procedures  we  will  use  and  what  labor 
standards  will  be  required.  That  being  stated,  it  goes  to  reason  that  we  can  show 
dependency  with  PBSA,  through  the  use  of  other  criteria. 

The  final  table  the  research  will  look  at  is  D1A  (Type  of  Entity  or  Business  Size). 
Like  that  of  B 12A  (Service  Code),  it  was  found  to  be  unstable  under  nominal  logistical 
regression,  but  merited  further  analysis  due  to  its  potential  weight  on  PBSA.  Despite  its 
previous  unstable  condition,  no  data  was  required  to  be  removed  in  order  to  conduct  this 
contingency  table  and  it  was  determined  that  there  exists  dependency  between  a 
contractor’s  size  and  that  of  PBSA.  As  previously  noted  through  demographics,  we 
know  that  the  bulk  of  awards  are  done  through  large  business.  However,  we  can  see  a 
predictive  value  across  all  business  concerns  with  no  anomalies  noted.  Thus,  we  know 
that  a  relationship  does  exit  between  these  two  variables. 
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Summary 

This  chapter  provided  a  discussion  of  the  data  results  that  were  obtained.  By 
taking  a  combination  of  the  three  analysis  tools  used,  an  insight  can  be  provided  as  to 
whether  PBSA  will  be  utilized  and  typically  where  it  has  been  used  in  the  past.  Areas 
have  been  identified,  through  demographics,  that  are  being  greatly  or  underutilized  and  to 
what  extent  each  play  in  the  overall  expenditure  of  PBSA.  Nominal  logistical  regression 
did  not  produce  any  dependencies,  but  dependency  among  variables  was  determined 
through  the  use  of  contingency  tables.  Chapter  5  will  discuss  conclusions  from  these 
results,  limitations  to  research,  and  possible  future  research  that  can  be  conducted. 
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V.  Conclusions  and  Recommendations 


The  purpose  of  this  research  was  to  provide  an  exploratory  study  within 
Perfonnance  Based  Service  Acquisition  (PBSA),  in  which  future  research  can  be  based. 
In  order  to  accomplish  this  task,  the  following  research  question  was  developed:  What  is 
the  status  of  PBSA  implementation  in  the  US  Air  Force  (USAF)?  The  following 
investigative  questions  were  developed  in  order  to  answer  the  research  question: 

1 .  Are  the  goals,  as  established  by  Dr.  Gansler  and  National  Defense  Authorization 
Act  (NDAA),  being  met  by  Air  Force  Material  Command  (AFMC)  and  the  US 
Air  Force  (USAF)  and  are  there  any  trends  to  suggest  future  usage  of  PBSA? 

2.  What  factors,  if  any,  are  related  to  whether  a  contract  is  classified  as  having  been 
awarded  using  PBSA  strategies? 

This  chapter  will  address  this  research  effort’s  conclusions,  benefits,  limitations,  and 
provide  suggestions  for  future  research. 

Conclusions 

Part  one  of  investigative  question  1,  has  the  USAF  met  PBSA  goals,  has  been 
answered  and  found  to  be  YES;  the  USAF  is  meeting  NDAA  FY  2002  requirements  of 
25%  of  service  actions  and  dollars  for  FY  2003  (NDAA,  2002).  While  the  NDAA  goals 
are  at  the  service  level,  based  on  the  research  found,  AFMC  is  not  meeting  the  intent  set 
by  the  NDAA,  given  the  two-year  potential  trend  in  data.  Also,  the  data  is  unlikely  to 
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make  the  following  year  requirement  of  35%  of  service  actions  and  dollars  (NDAA, 

2002). 

Part  two  of  investigative  question  1 ,  are  there  any  trends,  has  some  potentially 
troubling  news.  If  AFMC  continues  with  its  rate  of  PBSA  usage,  again  based  on  two 
years  of  data  analyzed,  the  USAF  will  find  it  difficult  to  make  the  FY  2004  NDAA 
requirement.  Since  the  requirements  by  Dr.  Gansler  and  the  NDAA  are  to  reach  50%  by 
FY  2005,  it  is  unlikely  the  remainder  of  the  USAF,  outside  of  AFMC,  will  be  able  to 
reach  or  maintain  higher  levels  then  they  are  already  at,  which  is  already  close  to  50%  for 
actions  and  slightly  above  50%  for  dollars  (Gansler,  2000;  NDAA,  2002).  Even  if  the 
organizations  outside  of  AFMC  do  obtain  higher  rates,  the  predominant  amount  of 
service  actions  and  dollars,  approximately  60%,  reside  within  AFMC  and  the  onus  will 
be  on  that  command  to  ensure  that  the  requirements  for  PBSA  are  being  met. 

Additional  trends  that  have  been  identified  through  demographics  can  be  utilized 
to  help  focus  efforts  for  future  usage.  It  is  this  researcher’s  opinion  that  key  types  of 
services  should  be  focused  upon  in  order  to  gain  the  optimal  amount  of  benefit  for  both 
AFMC  and  USAF.  This  would  include  all  services  that  have  been  indicated  as  greater 
then  1%  of  the  dollars  spent  for  services.  By  focusing  on  key  types  of  contracts,  such  as 
Firm  Fixed  Price  (FFP),  greater  efforts  can  be  placed  in  order  to  improve  their  usage 
within  PBSA.  FFP  is  one  of  the  primary  contract  types  for  utilizing  PBSA  and  yet  the 
USAF  is  only  at  44%  for  FFP.  In  the  cost  type  contract  arena,  specifications  can  be 
looked  at  in  order  to  see  if  contract  type  can  be  changed  and  support  PBSA.  In  addition 
to  focusing  on  the  four  indicated  contract  types  identified  in  chapter  4,  focusing  on  large 
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business  concerns  should  also  help  to  gain  the  greatest  benefits  for  trying  to  achieve 
established  goals. 

Investigative  question  2  asked  if  there  were  any  discemable  correlations  between 
whether  a  contract  is  awarded  PBSA  based  on  criteria  such  as  type  of  service,  dollar 
amount,  length  of  contract,  business  size,  and  contract  type.  It  is  unfortunate  that 
nominal  logistical  regression  was  unable  to  detennine  any  usable  correlations  or  patterns 
within  the  data,  in  order  to  develop  a  predictive  model.  It  was  the  hope  of  this  researcher 
to  build  a  model  in  were  if  certain  characteristics  were  known,  a  reasonable  prediction  as 
to  whether  a  contract  action  would  be  PBSA  or  not  could  be  determined.  Through  this 
effort,  characteristics  that  would  identify  more  with  a  No  response  to  PBSA  could  be 
analyzed  further  and  efforts  placed  in  those  areas. 

Despite  the  lack  of  insight  provided  by  nominal  logistical  regression,  contingency 
tables  provided  a  different  perspective.  Through  contingency  tables,  this  researcher 
identified  dependency  between  14  different  variables  or  sections  within  a  contract  action. 
By  identifying  these  dependencies,  we  gain  further  understanding  as  to  what  factors 
affect  PBSA  utilization.  The  primary  purpose  of  these  tables  was  to  detennine  if 
dependence  existed,  nothing  more.  This  tells  us  that  a  casual  relationship  may  exist,  but 
not  to  what  extent  the  dependency  is.  By  definition,  if  the  two  factors  are  dependent,  the 
occurrence  of  one  altars  the  probability  of  the  other  (McClave  et.al.,  2001). 

In  order  to  best  utilize  this  infonnation,  individuals  focused  on  using  PBSA  to  its 
greatest  extent,  should  look  for  these  common  factors.  If  these  factors  exist  within  a 
contract  action  and  it  is  not  identified  as  PBSA,  then  there  exists  the  possibility  for 
change.  Since  nominal  logistical  regression  is  inconclusive,  but  contingency  tables 
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identified  dependency,  by  understanding  the  two  we  can  focus  further  research  efforts  or 
focus  our  ability  to  use  these  individual  or  combined  factors  in  order  to  ensure  greater 
usage  of  PBS  A. 

Study  Limitations 

Certainly  there  are  always  ways  that  researchers  could  have  done  something 
better  throughout  their  work  and  this  project  is  no  exception.  This  researcher,  based  on 
time  constraints,  settled  for  the  data  that  was  provided  and  available  at  the  given  time.  If 
more  time  had  been  available,  additional  data  for  AFMC  and  USAF  could  have  been 
gathered  for  more  years  and  utilized  to  provide  an  added  in-depth  look. 

Based  on  this  lack  of  data  across  multiple  years,  the  results  of  this  data  are  skewed 
as  to  the  two-year  potential  trends  that  could  be  developed  across  AFMC  and 
comparisons  between  AFMC  and  USAF  for  one  year.  Multiple  years  of  data  could  have 
provided  a  different  insight  as  to  possible  cyclical  patterns  or  possible  predictive  models 
that  could  have  been  developed.  This  correlates  to  the  results  only  being  as  strong  as  the 
data  provided  and  could  have  a  different  outcome  if  the  data  had  been  greater. 

A  possible  second  limitation  to  this  research  is  the  approach  that  was  used  by  the 
researcher.  Since  very  little  research  has  been  done  in  this  arena,  it  was  the  determination 
of  this  researcher  to  perfonn  a  more  exploratory  study  as  a  building  block  for  further 
research  to  be  conducted.  The  data  suggests  that  this  type  of  research  is  not  the  most 
compatible  and  difficult  to  achieve.  If  the  research  had  been  approached  more  as  a  case 
study  or  as  a  survey,  it  is  possible  that  findings  would  have  come  to  different  conclusions 
and  gone  in  a  different  direction  then  this  research. 
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Future  Research 


There  are  several  possible  areas  that  can  be  utilized  for  future  research  following 
this  effort.  This  first,  and  most  obvious  one,  is  that  additional  data  across  multiple  years 
needs  to  be  gathered.  From  this,  a  greater  analysis  regarding  trends,  predictive  models, 
etc.,  can  hopefully  be  built  and  a  greater  insight  into  the  world  of  PBSA,  beyond  this 
researcher’s  efforts,  obtained. 

A  second  possible  research  area,  briefly  touched  upon  in  chapter  2,  is  that  of 
training.  Previous  research  from  the  RAND  Corporation’s  Project  AIR  FORCE, 
identified  specific  areas  that  needed  to  be  looked  at  when  viewing  PBSA,  one  of  which 
included  training  (Ausink  et.al.,  2001).  There  is  additional  literature  available  that 
supports  the  need  for  training  within  PBSA,  but  does  not  develop  the  extent  required  and 
no  research  has  been  discovered  that  has  investigated  this  region  of  PBSA.  This  area  has 
possible  merit  for  future  research  within  this  field. 

It  has  been  mentioned  various  times  that  the  USAF  should  apply  commercial 
industries’  best  practices  to  PBSA  (Ausink  et.al.,  2001;  Gansler,  2001;  Moore  et.al., 

2002;  OFPP,  1994).  However,  this  researcher  found  no  clear  cut  or  established  criteria  as 
to  what  these  commercial  best  practices  are.  It  is  understood  that  best  practices  will 
change  in  time,  but  what  are  the  current  best  practices  and  are  they  being  utilized.  It 
would  be  of  some  value  if  a  snap  shot  of  what  industry  considers  its  best  practices  were 
determined  and  then  compared  to  the  USAF,  in  order  to  determine  if  these  best  practices 
are  being  gathered. 
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The  final  area  to  recommend  for  research  involves  the  commercial  sector.  It  was 


referenced  toward  the  end  of  chapter  2  that  industry  only  utilizes  PBSA  between  5-20% 
of  its  services  acquisition  efforts  (Barry,  2003).  It  would  be  of  interest  if  this  was 
examined  further  and  a  determination  made  as  to  the  true  extent  of  PBSA  usage  within 
industry.  Based  on  these  findings,  a  comparison  could  then  be  accomplished  against  the 
USAF  and  DoD.  Additionally,  the  requirements  that  are  set  forth  for  PBSA  could  be 
reviewed  and  based  on  comparison,  a  determination  made  as  to  whether  they  are 
reasonable  or  not.  Final  conclusions  could  then  be  drawn  as  to  whether  the  DOD  and 
USAF  are  heading  down  the  appropriate  road  or  these  requirements  need  to  be  re¬ 
evaluated  at  the  service  secretary  level. 

Conclusion 

This  chapter  provided  a  brief  summary  of  this  research  effort  and  conclusions  that 
could  be  drawn  from  this  research.  Areas  addressed  included  the  benefits  of  this 
research,  the  research’s  limitations,  and  suggestions  for  future  research. 
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